diff --git a/docs/networking/network.md b/docs/networking/network.md index 641e7df1..a8ef44e6 100644 --- a/docs/networking/network.md +++ b/docs/networking/network.md @@ -306,8 +306,43 @@ host. It never decrypts TLS, so it holds no signing key. It caches nothing. the canary widens the restricted list by that one name. The swarm's own service names aren't on it: they resolve to the bridge or a private address, which the rules below refuse, and agents reach them directly. Each entry - matches that exact name. A user on the unrestricted list reaches any + matches that exact name, or with a leading dot the name and its subdomains. A user on the unrestricted list reaches any other name too, on ports 80 and 443. Everyone else gets a 403 for it. + `allowedDomains` is the hive's own list. Squid uses it until the swarm has a + list of its own (next bullet). +- **Swarm allow-list.** The operator adds domains in the swarm UI + ([allowed domains](../swarm/ui.md#allowed-domains)); the swarm controller + keeps them in one value of the `egress-allow` bucket, each for every agent + or for one agent. Every 30 seconds `hive-c0re` reads it and writes two files + beside the users file, `services.hyperhive.squid.allowedFile` + (`/var/lib/hive-squid/allowed`) and `services.hyperhive.squid.agentAllowFile` + (`/var/lib/hive-squid/agent-allow.conf`), `hive-core:squid`, `0640`, and + logs `squid allow list: 2 swarm, 3 agent entries` when they change. The + swarm-wide names join the allow ACL, so a restricted user reaches them. A + per-agent entry is a rule for that agent's squid user, `http_access allow + `, in the generated `agent-allow.conf` that the configuration + includes at the point where the base list gets its allow: after every + destination deny and + `deny !authed`, so the rules under _Denied before any allow_ still bind it, + and before the unrestricted rule. Only this hive's agents get a rule. + `squid-unrestricted-reload.path` reloads squid when either file changes, so + there is no deploy or restart. + A name is a DNS name of at least two labels. One leading dot (`.example.com`) + also allows every subdomain, and is the only wildcard; the controller refuses + IP addresses, other wildcards, ports, paths and the base list's names when + you add the entry, and `hive-c0re` checks each name again and skips one that + fails, or that the base list or the canary covers, or that another entry of + the same list covers. A name that resolves to a private address stays a 403. + The swarm's list replaces the hive's `allowedDomains` as soon as it exists, + even when empty: add the names the hive allowed to the swarm's list + before relying on it. Until the controller has a list (it predates the + feature, or nobody added an entry) `allowedFile` holds `allowedDomains` and + `agent-allow.conf` is empty. A queue that stops answering for the bucket + leaves a file the swarm wrote untouched, and a failed read leaves both + files untouched. The first line of a file written from the swarm's list is a + `# source: swarm` comment. The queue grant a hive needs to read the list + comes with the controller's `swarm-nats-auth`; a hive that can't read it + times out, keeps its files and warns after about 90 seconds. - **Unlisted names.** For a user who isn't on the unrestricted list, refused after the destination checks, which look the name up on the host. - **Denied before any allow.** Destinations that resolve to loopback, diff --git a/docs/swarm/ui.md b/docs/swarm/ui.md index 3aa7d3b9..8df40093 100644 --- a/docs/swarm/ui.md +++ b/docs/swarm/ui.md @@ -112,7 +112,7 @@ with no icon keeps the hyperhive mark. The tab also has an agent badge, with the agent's icon and name, ahead of the connection badge. It opens the agent's details in a popover under the badge: -the same status, state, wanted state, inference settings, resource caps, permissions and linked accounts as the +the same status, state, wanted state, inference settings, resource caps, permissions, allowed domains and linked accounts as the panel on `/agents`, without the terminal preview. The popover doesn't dim the page. Selecting outside it, or Escape, closes it. From the keyboard, opening a popover moves focus into it, and Escape puts focus back on the badge. The filter @@ -622,6 +622,38 @@ restarts the agent after its current turn. The panel reads the reason and a **retry** button. `swarmctl agent permissions set` sets the same grant from a shell. +### Allowed domains + +The agent's detail panel lists the destinations it may reach through its +hive's egress proxy under **allowed domains**, beyond the base list every +agent has: the entries for every agent in the swarm (badge **swarm-wide**) and +the ones for this agent alone (badge with the agent's name). An entry matters +to a restricted agent. One with the unrestricted internet flag reaches any +name the proxy's private-range rules leave. **remove** drops one entry; +**add** opens a dialog with a domain box and a tick for every agent in the +swarm. The tick is off by default, so an entry is for this agent alone. + +A domain is a DNS name of at least two labels, such as `github.com`. One +leading dot, as in `.githubusercontent.com`, also allows every subdomain; the +dot form is the only wildcard, because exact names can't list a service that +spreads over many hostnames, and an entry is the operator's choice. The +controller refuses IP addresses, other wildcards, ports and paths, and a name +the proxy already allows every agent (the base list). The dialog shows the +refusal's reason under the box and leaves what you typed. Adding does a +`POST /api/egress/allowed-domains` and removing a `DELETE` of the same path, +each with `{"domain": "github.com", "agent": "atlas"}` (leave out `agent` for +a swarm-wide entry); both answer the whole list, and `GET` reads it. Adding +an entry that exists changes nothing. + +A saved change isn't instant. Each hive reads the list about every 30 +seconds and reloads its proxy, so the name is reachable within a minute. The +proxy's private-range rules still apply: an entry whose name resolves to a +loopback, private or bridge address stays blocked. The first entry makes the +swarm's list replace the names a hive's own `allowedDomains` lists, so add +those names here too; a hive keeps its own list until then. See [the egress +proxy](../networking/network.md#egress-proxy-squid) for what each hive does +when the controller is old or unreachable. + ### Creating an agent The **+ agent** button opens a dialog of five steps, which you go through in diff --git a/frontend/packages/swarm-ui/src/agent/detail/AgentDetail.tsx b/frontend/packages/swarm-ui/src/agent/detail/AgentDetail.tsx index 9531d832..382688d1 100644 --- a/frontend/packages/swarm-ui/src/agent/detail/AgentDetail.tsx +++ b/frontend/packages/swarm-ui/src/agent/detail/AgentDetail.tsx @@ -14,6 +14,7 @@ import { AgentInference } from "./AgentInference.js"; import { AgentInstructions } from "./AgentInstructions.js"; import { AgentLimits } from "./AgentLimits.js"; import { AgentPermissions } from "./AgentPermissions.js"; +import { AllowedDomains } from "./AllowedDomains.js"; import { AgentStateRow } from "./AgentStateRow.js"; import { AgentTermPreview } from "../terminal/AgentTermPreview.js"; import { FRESHNESS, type AgentRow } from "../AgentTypes.js"; @@ -122,6 +123,10 @@ export function AgentDetail({
+
allowed domains
+
+ +
accounts
{row.hive ? ( diff --git a/frontend/packages/swarm-ui/src/agent/detail/AllowedDomains.tsx b/frontend/packages/swarm-ui/src/agent/detail/AllowedDomains.tsx new file mode 100644 index 00000000..8d690b59 --- /dev/null +++ b/frontend/packages/swarm-ui/src/agent/detail/AllowedDomains.tsx @@ -0,0 +1,243 @@ +// — the destinations the swarm lets one agent reach through +// its hive's egress proxy, beyond the base list every agent has: the entries +// that hold for every agent and the ones for this agent alone, with an editor +// to add and remove them. +// +// Reads `GET /api/egress/allowed-domains`, which answers every entry, and +// shows the ones that apply to this agent. Adding POSTs one entry and removing +// DELETEs one, each with the entry as the body, each answering the whole list. +// The route's refusal (an IP address, a wildcard, a name the proxy already +// allows everyone) shows as it arrives, next to the box. The hive reads a +// change within about 30 seconds. +// +// The first entry makes the swarm's list replace the names the hive's own +// configuration allows, so a name the hive allowed and the swarm's list lacks +// stops being allowed once hives read it. An agent that holds the unrestricted +// internet flag reaches any name the proxy's private-range rules leave, so an +// entry matters only to a restricted one. +// +// Its state belongs to one agent: callers key it by agent name. +import { useEffect, useState } from "preact/hooks"; +import { ApiErrorPanel } from "@hive/shared/api-error-panel.js"; +import { readApiError, type ProblemDetails } from "@hive/shared/api-error.js"; +import { Badge } from "@hive/shared/badge.js"; +import { Button } from "../../ui/button/Button.js"; +import { CheckboxField } from "../../ui/checkbox-field/CheckboxField.js"; +import { Dialog } from "../../ui/dialog/Dialog.js"; +import { Panel } from "../../ui/panel/Panel.js"; +import { TextField } from "../../ui/text-field/TextField.js"; +import { + ALLOWED_DOMAINS_URL, + allowedSummary, + entriesFor, + entryBody, + type AllowedDomain, +} from "./allowed-domains.js"; +import "./AgentLimits.css"; + +interface Listing { + entries: AllowedDomain[]; +} + +export function AllowedDomains({ agent }: { agent: string }) { + const [entries, setEntries] = useState(null); + const [error, setError] = useState(null); + // Bumped after an add or a removal, so the list is fetched again. + const [reload, setReload] = useState(0); + const [adding, setAdding] = useState(false); + const [busy, setBusy] = useState(false); + + useEffect(() => { + let cancelled = false; + (async () => { + const r = await fetch(ALLOWED_DOMAINS_URL); + if (!r.ok) { + if (!cancelled) setError(await readApiError(r)); + return; + } + const body = (await r.json()) as Listing; + if (cancelled) return; + setEntries(body.entries); + setError(null); + })().catch((e: unknown) => { + if (!cancelled) setError({ detail: String(e) }); + }); + return () => { + cancelled = true; + }; + }, [reload]); + + async function remove(entry: AllowedDomain) { + if (busy) return; + setBusy(true); + try { + const r = await fetch(ALLOWED_DOMAINS_URL, { + method: "DELETE", + headers: { "content-type": "application/json" }, + body: JSON.stringify(entry), + }); + if (!r.ok) { + setError(await readApiError(r)); + return; + } + setReload((n) => n + 1); + } catch (e: unknown) { + setError({ detail: String(e) }); + } finally { + setBusy(false); + } + } + + // A list already loaded stays under a failed refetch, so a change that + // worked is not hidden by a read that did not. + const errorPanel = error ? ( + <> + + + + ) : null; + if (entries === null) { + return ( + errorPanel ?? ( + + loading the allowed domains… + + ) + ); + } + + const shown = entriesFor(entries, agent); + return ( +
+ {errorPanel} + + {shown.length > 0 ? ( +
    + {shown.map((e) => ( +
  • + {" "} + void remove(e)} + /> +
    • + ))} +
+ ) : null} + setAdding(true)} + /> + setAdding(false)} + label="add an allowed domain" + plain + > + {/* Remounted per open, so a cancelled add leaves nothing behind. */} + {adding ? ( + setAdding(false)} + onAdded={() => { + setAdding(false); + setReload((n) => n + 1); + }} + /> + ) : null} + +
+ ); +} + +function AddForm({ + agent, + onClose, + onAdded, +}: { + agent: string; + onClose: () => void; + onAdded: () => void; +}) { + const [domain, setDomain] = useState(""); + const [forSwarm, setForSwarm] = useState(false); + const [busy, setBusy] = useState(false); + const [problem, setProblem] = useState(null); + + async function save(e: Event) { + e.preventDefault(); + setBusy(true); + setProblem(null); + try { + const r = await fetch(ALLOWED_DOMAINS_URL, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify(entryBody(domain, forSwarm, agent)), + }); + if (!r.ok) { + // The box stays as typed, so the operator can fix it and add again. + setProblem(await readApiError(r)); + return; + } + onAdded(); + } catch (err: unknown) { + setProblem({ detail: String(err) }); + } finally { + setBusy(false); + } + } + + return ( + +

+ Lets {agent} reach one more name through its hive's proxy, or every + agent when the box below is ticked. A name needs at least two labels, + such as github.com; a leading dot, as in{" "} + .githubusercontent.com, also allows every subdomain. IP + addresses and wildcards are refused. A name that resolves to a private + address stays blocked. The first entry makes the swarm's list replace + the names each hive's own configuration allows. The hive reads a change + within about 30 seconds. +

+
void save(e)}> + + + {problem ? ( + + ) : null} +
+ +
+ +
+ ); +} diff --git a/frontend/packages/swarm-ui/src/agent/detail/allowed-domains.ts b/frontend/packages/swarm-ui/src/agent/detail/allowed-domains.ts new file mode 100644 index 00000000..75820332 --- /dev/null +++ b/frontend/packages/swarm-ui/src/agent/detail/allowed-domains.ts @@ -0,0 +1,52 @@ +// The detail panel's allowed-domains editor: which entries one agent's panel +// shows and the bodies it sends. Kept out of the component so `npm test` can +// run it. + +/** One entry as `GET /api/egress/allowed-domains` carries it. */ +export interface AllowedDomain { + domain: string; + /** Absent for an entry that holds for every agent in the swarm. */ + agent?: string; +} + +export const ALLOWED_DOMAINS_URL = "/api/egress/allowed-domains"; + +/** + * The entries that apply to `agent`: the swarm-wide ones first, then the ones + * for this agent alone. Entries for other agents are not shown on this panel. + */ +export function entriesFor( + entries: AllowedDomain[], + agent: string, +): AllowedDomain[] { + const swarmWide = entries.filter((e) => e.agent === undefined); + const own = entries.filter((e) => e.agent === agent); + return [...swarmWide, ...own]; +} + +/** What the panel says about a list of entries. */ +export function allowedSummary(shown: AllowedDomain[]): string { + if (shown.length === 0) return "none added"; + const swarmWide = shown.filter((e) => e.agent === undefined).length; + const own = shown.length - swarmWide; + return [ + swarmWide > 0 ? `${swarmWide} swarm-wide` : null, + own > 0 ? `${own} for this agent` : null, + ] + .filter((part) => part !== null) + .join(", "); +} + +/** + * The body that adds or removes `domain`, as typed: the route lower-cases it + * and refuses what is not a name. `forSwarm` makes it hold for every agent; + * otherwise it is for `agent` alone. + */ +export function entryBody( + domain: string, + forSwarm: boolean, + agent: string, +): AllowedDomain { + const name = domain.trim(); + return forSwarm ? { domain: name } : { domain: name, agent }; +} diff --git a/frontend/packages/swarm-ui/test/allowed-domains.test.mjs b/frontend/packages/swarm-ui/test/allowed-domains.test.mjs new file mode 100644 index 00000000..8cda747b --- /dev/null +++ b/frontend/packages/swarm-ui/test/allowed-domains.test.mjs @@ -0,0 +1,54 @@ +// The detail panel's allowed-domains editor: which entries one agent sees and +// the bodies it sends. Run by `npm test` with Node's type stripping. +import assert from "node:assert/strict"; +import { test } from "node:test"; +import { + allowedSummary, + entriesFor, + entryBody, +} from "../src/agent/detail/allowed-domains.ts"; + +const all = [ + { domain: "crates.io", agent: "atlas" }, + { domain: "docs.rs", agent: "argus" }, + { domain: "github.com" }, + { domain: "pypi.org", agent: "atlas" }, +]; + +test("an agent sees the swarm-wide entries first, then its own, and not another agent's", () => { + assert.deepEqual(entriesFor(all, "atlas"), [ + { domain: "github.com" }, + { domain: "crates.io", agent: "atlas" }, + { domain: "pypi.org", agent: "atlas" }, + ]); + // Control: the other agent's entry is there to be found. + assert.deepEqual(entriesFor(all, "argus"), [ + { domain: "github.com" }, + { domain: "docs.rs", agent: "argus" }, + ]); + assert.deepEqual(entriesFor(all, "iris"), [{ domain: "github.com" }]); + assert.deepEqual(entriesFor([], "atlas"), []); +}); + +test("the summary counts each scope", () => { + assert.equal(allowedSummary([]), "none added"); + assert.equal( + allowedSummary(entriesFor(all, "atlas")), + "1 swarm-wide, 2 for this agent", + ); + assert.equal(allowedSummary(entriesFor(all, "iris")), "1 swarm-wide"); + assert.equal( + allowedSummary([{ domain: "a.io", agent: "atlas" }]), + "1 for this agent", + ); +}); + +test("the body is swarm-wide or for the agent, with the name trimmed", () => { + assert.deepEqual(entryBody(" github.com ", true, "atlas"), { + domain: "github.com", + }); + assert.deepEqual(entryBody("github.com", false, "atlas"), { + domain: "github.com", + agent: "atlas", + }); +}); diff --git a/hive-c0re/Cargo.toml b/hive-c0re/Cargo.toml index fa1713df..12f36caa 100644 --- a/hive-c0re/Cargo.toml +++ b/hive-c0re/Cargo.toml @@ -62,6 +62,7 @@ swarm-queue-client = { workspace = true, features = [ "kv", "notices", "agent-limits", + "egress-allow", "agent-permissions", "agent-archetype", "agent-closures", diff --git a/hive-c0re/src/swarm_status.rs b/hive-c0re/src/swarm_status.rs index 71cdafa1..d8816ea7 100644 --- a/hive-c0re/src/swarm_status.rs +++ b/hive-c0re/src/swarm_status.rs @@ -116,6 +116,12 @@ pub fn spawn( shutdown.clone(), )); + // The swarm's egress allow-list, on the same connection. + tokio::spawn(crate::workers::squid_allow::follow( + client.clone(), + shutdown.clone(), + )); + // The permissions the swarm grants, and the import of the meta repo's. tokio::spawn(crate::workers::swarm_permissions::follow( client.clone(), diff --git a/hive-c0re/src/workers/mod.rs b/hive-c0re/src/workers/mod.rs index 87da7e4e..4915fdb2 100644 --- a/hive-c0re/src/workers/mod.rs +++ b/hive-c0re/src/workers/mod.rs @@ -11,6 +11,7 @@ pub mod crash_watch; pub mod knowledge; pub mod mcp_sockets; pub mod permissions_import; +pub mod squid_allow; pub mod squid_users; pub mod swarm_limits; pub mod swarm_permissions; diff --git a/hive-c0re/src/workers/squid_allow.rs b/hive-c0re/src/workers/squid_allow.rs new file mode 100644 index 00000000..c3c95bce --- /dev/null +++ b/hive-c0re/src/workers/squid_allow.rs @@ -0,0 +1,596 @@ +//! Squid's allow-list files: the destination names the swarm lets this hive's +//! agents reach through the egress proxy. +//! +//! swarm-controller keeps one list for the whole swarm +//! ([`swarm_wire::egress_allow`]); each entry is for every agent or for one. +//! Every [`INTERVAL`] this reads it and renders two files for squid: +//! +//! - [`ENV_ALLOWED_FILE`]: the swarm-wide names, one per line, which +//! `nix/host-modules/hive-squid.nix` reads as part of `allowed_dst`. +//! - [`ENV_AGENT_ALLOW_FILE`]: squid configuration `include`d where the base +//! list is allowed: for each of this hive's agents with entries, an `acl` +//! for its squid user, an `acl` for its names and the `http_access allow` +//! joining them. Entries of agents this hive does not run are left out. +//! +//! Squid reads both when it parses its configuration, so the path unit in +//! `nix/host-modules/hive-squid.nix` reloads it when either changes. +//! +//! The swarm's list replaces the names `services.hyperhive.squid.allowedDomains` +//! lists ([`ENV_SEED_FILE`]), which stay the list until the swarm has one. A +//! file rendered from the swarm's list starts with [`SWARM_MARKER`]. The queue +//! not answering for the bucket does not show that the swarm withdrew its list, +//! so a file with the marker is left alone then. +//! +//! Every entry is checked again with +//! [`swarm_wire::egress_allow::normalize_domain`], since the names end up in +//! squid's configuration, and skipped when it fails. An entry that +//! [`ENV_ALWAYS_ALLOWED`] or another entry of its list covers is dropped, +//! because squid ignores the narrower of two overlapping names in one ACL. +//! +//! A failed read leaves both files as they are. + +use std::path::{Path, PathBuf}; +use std::time::Duration; + +use anyhow::{Context, Result}; +use swarm_queue_client::egress_allow::{self, Read}; +use swarm_wire::egress_allow::{AllowList, covered_by, normalize_domain}; + +use super::squid_users::write_if_changed; +use crate::stats::sweep_health::{self, SweepHealth}; + +/// How often the list is read. +pub const INTERVAL: Duration = Duration::from_secs(30); + +/// At [`INTERVAL`] this is ~90 seconds of failing reads. +const FAILURES_BEFORE_WARNING: u32 = 3; + +/// The swarm-wide names file. Set by `nix/host-modules/hive-c0re` from +/// `services.hyperhive.squid.allowedFile`. +pub const ENV_ALLOWED_FILE: &str = "HIVE_SQUID_ALLOWED_FILE"; + +/// The per-agent squid configuration file. Set from +/// `services.hyperhive.squid.agentAllowFile`. +pub const ENV_AGENT_ALLOW_FILE: &str = "HIVE_SQUID_AGENT_ALLOW_FILE"; + +/// The file of names `services.hyperhive.squid.allowedDomains` lists, one per +/// line, which the files hold until the swarm has a list. +pub const ENV_SEED_FILE: &str = "HIVE_SQUID_ALLOWED_SEED_FILE"; + +/// The names, separated by spaces, that squid allows every authenticated user +/// regardless of this list: the base list and the self-check canary. +pub const ENV_ALWAYS_ALLOWED: &str = "HIVE_SQUID_ALWAYS_ALLOWED"; + +/// The first line of a file rendered from the swarm's list. +const SWARM_MARKER: &str = "# source: swarm\n"; + +/// The first line of a file rendered from the seed. +const SEED_MARKER: &str = "# source: seed\n"; + +/// The two files a pass writes. +struct Files<'a> { + allowed: &'a Path, + agent_allow: &'a Path, +} + +/// What a pass left in the files. +#[derive(Debug, PartialEq, Eq)] +enum Source { + /// The swarm's list. + Swarm, + /// The seed, since the swarm has no list. + Seed, + /// The swarm has no list now, but the files hold the one it had. + Kept, +} + +fn env_path(name: &str) -> Option { + std::env::var_os(name) + .filter(|v| !v.is_empty()) + .map(PathBuf::from) +} + +/// Read the list now and every [`INTERVAL`] until shutdown, when this host has +/// squid's two files. +pub async fn follow(client: async_nats::Client, mut shutdown: tokio::sync::watch::Receiver) { + let (Some(allowed), Some(agent_allow)) = + (env_path(ENV_ALLOWED_FILE), env_path(ENV_AGENT_ALLOW_FILE)) + else { + return; + }; + let seed = env_path(ENV_SEED_FILE); + let always = std::env::var(ENV_ALWAYS_ALLOWED).unwrap_or_default(); + let mut health = SweepHealth::new("squid_allow_render", "warn", FAILURES_BEFORE_WARNING); + // The last `Source` that was logged, so a list that stays unset is + // announced once rather than every pass. + let mut announced: Option<&'static str> = None; + loop { + let files = Files { + allowed: &allowed, + agent_allow: &agent_allow, + }; + match pass(&client, &files, seed.as_deref(), &always).await { + Ok(source) => { + health.record_ok(); + announce(&source, &mut announced); + } + Err(e) => { + tracing::warn!(error = ?e, "squid allow list: the pass failed"); + let err = format!("{e:#}"); + health.record_err(|ctx| { + let age = ctx.since_last_ok.map_or_else( + || "no success this session".to_owned(), + |d| format!("last ok {} ago", sweep_health::fmt_age(d)), + ); + format!( + "reading the swarm's egress allow-list is failing ({} consecutive, \ + {age}) — squid keeps the list last written: {err}", + ctx.consecutive + ) + }); + } + } + tokio::select! { + () = tokio::time::sleep(INTERVAL) => {} + _ = shutdown.changed() => return, + } + } +} + +/// Log the first pass of a run that left the seed or the kept files. +fn announce(source: &Source, announced: &mut Option<&'static str>) { + let (key, message) = match source { + Source::Swarm => { + *announced = None; + return; + } + Source::Seed => ( + "seed", + "squid allow list: the swarm has no list, so squid keeps this hive's own names", + ), + Source::Kept => ( + "kept", + "squid allow list: the swarm's list is not readable now, so squid keeps the last one written", + ), + }; + if *announced != Some(key) { + tracing::info!("{message}"); + *announced = Some(key); + } +} + +async fn pass( + client: &async_nats::Client, + files: &Files<'_>, + seed: Option<&Path>, + always: &str, +) -> Result { + // An unconnected client does not fail a JetStream request, it hangs on it. + swarm_queue_client::ensure_connected(client)?; + let agents: Vec = crate::lifecycle::agents_for_meta_listing() + .await + .context("enumerating this hive's agents")? + .into_iter() + .map(|spec| spec.name) + .collect(); + let read = egress_allow::read(client) + .await + .map_err(|e| anyhow::anyhow!(e)) + .context("reading the swarm's egress allow-list")?; + sync_with(read, &agents, seed, always, files) +} + +/// [`pass`] with the read, the agents and the files given. Each file is only +/// written when its rendering differs from what it holds. +fn sync_with( + read: Read, + agents: &[String], + seed: Option<&Path>, + always: &str, + files: &Files<'_>, +) -> Result { + let always: Vec<&str> = always.split_whitespace().collect(); + let rendered = match read { + Read::Current(list) => render(&list, agents, &always), + Read::NotSet => { + if super::squid_users::read_or_empty(files.allowed)?.starts_with(SWARM_MARKER) { + return Ok(Source::Kept); + } + seeded(seed)? + } + }; + let changed_allowed = write_if_changed(files.allowed, &rendered.allowed)?; + let changed_agents = write_if_changed(files.agent_allow, &rendered.agent_allow)?; + if changed_allowed || changed_agents { + if let Some(counts) = &rendered.counts { + tracing::info!(file = %files.allowed.display(), "{}", counts.summary()); + if !counts.skipped.is_empty() { + tracing::warn!( + skipped = ?counts.skipped, + "squid allow list: entries the swarm holds were skipped as invalid" + ); + } + } else { + tracing::info!( + file = %files.allowed.display(), + "squid allow list: wrote this hive's own names, no swarm list" + ); + } + } + Ok(if rendered.counts.is_some() { + Source::Swarm + } else { + Source::Seed + }) +} + +/// What the swarm's list rendered as. +struct Counts { + /// Swarm-wide names written. + swarm: usize, + /// Names written across this hive's agents. + agent: usize, + /// Entries left out because they are not valid names or agents. + skipped: Vec, +} + +impl Counts { + /// The info line a pass that changed a file logs. + fn summary(&self) -> String { + format!( + "squid allow list: {} swarm, {} agent entries", + self.swarm, self.agent + ) + } +} + +struct Rendered { + allowed: String, + agent_allow: String, + /// `None` for the seed. + counts: Option, +} + +/// The files for the seed: its names as they are, and no per-agent rules. +fn seeded(seed: Option<&Path>) -> Result { + let names = match seed { + Some(path) => super::squid_users::read_or_empty(path)?, + None => String::new(), + }; + let mut allowed = SEED_MARKER.to_owned(); + for line in names.lines().map(str::trim) { + if !line.is_empty() && !line.starts_with('#') { + allowed.push_str(line); + allowed.push('\n'); + } + } + Ok(Rendered { + allowed, + agent_allow: SEED_MARKER.to_owned(), + counts: None, + }) +} + +/// `names` sorted, with each one that another covers dropped. +fn reduced(mut names: Vec) -> Vec { + names.sort(); + names.dedup(); + let all = names.clone(); + names.retain(|n| { + let others: Vec<&String> = all.iter().filter(|o| *o != n).collect(); + !covered_by(n, &others) + }); + names +} + +/// The files for `list`, for the agents this hive runs. +fn render(list: &AllowList, agents: &[String], always: &[&str]) -> Rendered { + let mut skipped = Vec::new(); + let mut swarm_wide = Vec::new(); + let mut per_agent: std::collections::BTreeMap<&str, Vec> = + std::collections::BTreeMap::new(); + for entry in &list.entries { + let Ok(domain) = normalize_domain(&entry.domain) else { + skipped.push(entry.domain.clone()); + continue; + }; + match entry.agent.as_deref() { + None => swarm_wide.push(domain), + Some(agent) => { + if hive_types::Ident::parse(agent).is_err() { + skipped.push(format!("{} for {agent}", entry.domain)); + } else if let Some(mine) = agents.iter().find(|a| *a == agent) { + per_agent.entry(mine.as_str()).or_default().push(domain); + } + } + } + } + swarm_wide.retain(|d| !covered_by(d, always)); + let swarm_wide = reduced(swarm_wide); + + let mut allowed = SWARM_MARKER.to_owned(); + for name in &swarm_wide { + allowed.push_str(name); + allowed.push('\n'); + } + let mut agent_rules = Vec::new(); + let mut agent_count = 0; + for (agent, mut names) in per_agent { + names.retain(|d| !covered_by(d, always) && !covered_by(d, &swarm_wide)); + let names = reduced(names); + if names.is_empty() { + continue; + } + agent_count += names.len(); + agent_rules.push(format!( + "acl agent_allow_{agent} proxy_auth {agent}\n\ + acl agent_allow_{agent}_dst dstdomain {}\n\ + http_access allow agent_allow_{agent} agent_allow_{agent}_dst\n", + names.join(" ") + )); + } + Rendered { + allowed, + agent_allow: format!("{SWARM_MARKER}{}", agent_rules.concat()), + counts: Some(Counts { + swarm: swarm_wide.len(), + agent: agent_count, + skipped, + }), + } +} + +#[cfg(test)] +mod tests { + use super::*; + use swarm_wire::egress_allow::Entry; + + fn entry(domain: &str, agent: Option<&str>) -> Entry { + Entry { + domain: domain.to_owned(), + agent: agent.map(str::to_owned), + } + } + + fn list(entries: &[(&str, Option<&str>)]) -> AllowList { + AllowList { + entries: entries.iter().map(|(d, a)| entry(d, *a)).collect(), + } + } + + fn agents(names: &[&str]) -> Vec { + names.iter().map(|n| (*n).to_owned()).collect() + } + + const ALWAYS: &[&str] = &["api.anthropic.com", ".claude.ai", "cache.nixos.org"]; + + fn rendered(entries: &[(&str, Option<&str>)], mine: &[&str]) -> Rendered { + render(&list(entries), &agents(mine), ALWAYS) + } + + #[test] + fn swarm_wide_names_are_sorted_one_per_line_after_the_marker() { + let r = rendered( + &[ + ("github.com", None), + ("crates.io", None), + ("GitHub.com", None), + ], + &["atlas"], + ); + assert_eq!(r.allowed, "# source: swarm\ncrates.io\ngithub.com\n"); + let counts = r.counts.expect("a swarm list"); + assert_eq!((counts.swarm, counts.agent), (2, 0)); + } + + #[test] + fn an_empty_list_renders_an_empty_set_of_names_and_no_rules() { + let r = rendered(&[], &["atlas"]); + assert_eq!(r.allowed, SWARM_MARKER); + assert_eq!(r.agent_allow, SWARM_MARKER); + let counts = r.counts.expect("an empty swarm list is still a swarm list"); + assert_eq!((counts.swarm, counts.agent), (0, 0)); + } + + #[test] + fn an_agents_names_become_a_user_rule_and_other_hives_agents_are_left_out() { + let r = rendered( + &[ + ("pypi.org", Some("atlas")), + ("crates.io", Some("atlas")), + ("github.com", Some("iris")), + ("github.com", Some("../argus")), + ], + &["atlas", "argus"], + ); + assert_eq!( + r.agent_allow, + "# source: swarm\n\ + acl agent_allow_atlas proxy_auth atlas\n\ + acl agent_allow_atlas_dst dstdomain crates.io pypi.org\n\ + http_access allow agent_allow_atlas agent_allow_atlas_dst\n" + ); + let counts = r.counts.expect("a swarm list"); + assert_eq!((counts.swarm, counts.agent), (0, 2)); + // The name that could break out of the line is reported, not rendered. + assert_eq!(counts.skipped, ["github.com for ../argus"]); + assert!(!r.agent_allow.contains("iris"), "another hive's agent"); + assert!(!r.agent_allow.contains("argus"), "an invalid name"); + } + + #[test] + fn an_entry_that_is_not_a_name_never_reaches_squid_configuration() { + let r = rendered( + &[ + ("github.com", None), + ("1.2.3.4", None), + ("*.example.com", Some("atlas")), + ("example.com\nhttp_access allow all", Some("atlas")), + ("example.com\nhttp_access allow all", None), + ], + &["atlas"], + ); + assert_eq!(r.allowed, "# source: swarm\ngithub.com\n"); + assert_eq!(r.agent_allow, SWARM_MARKER); + assert!(!r.allowed.contains("http_access") && !r.agent_allow.contains("http_access")); + assert_eq!(r.counts.expect("a swarm list").skipped.len(), 4); + } + + #[test] + fn a_name_squid_allows_everyone_or_the_swarm_already_lists_is_dropped() { + let r = rendered( + &[ + ("api.anthropic.com", None), + ("a.claude.ai", None), + ("cache.nixos.org", Some("atlas")), + (".github.com", None), + ("api.github.com", None), + ("api.github.com", Some("atlas")), + ("crates.io", Some("atlas")), + ], + &["atlas"], + ); + assert_eq!(r.allowed, "# source: swarm\n.github.com\n"); + assert_eq!( + r.agent_allow, + "# source: swarm\n\ + acl agent_allow_atlas proxy_auth atlas\n\ + acl agent_allow_atlas_dst dstdomain crates.io\n\ + http_access allow agent_allow_atlas agent_allow_atlas_dst\n" + ); + // Control: the same entries survive when nothing is always allowed. + let bare = render( + &list(&[("api.anthropic.com", None), ("a.claude.ai", None)]), + &agents(&["atlas"]), + &[], + ); + assert_eq!( + bare.allowed, + "# source: swarm\na.claude.ai\napi.anthropic.com\n" + ); + } + + #[test] + fn the_log_line_carries_the_counts() { + let r = rendered( + &[ + ("github.com", None), + ("crates.io", None), + ("pypi.org", Some("atlas")), + ("npmjs.org", Some("atlas")), + ("docs.rs", Some("argus")), + ], + &["atlas", "argus"], + ); + let counts = r.counts.expect("a swarm list"); + assert_eq!( + counts.summary(), + "squid allow list: 2 swarm, 3 agent entries" + ); + } + + struct Dir(tempfile::TempDir); + + impl Dir { + fn allowed(&self) -> PathBuf { + self.0.path().join("allowed") + } + fn agent_allow(&self) -> PathBuf { + self.0.path().join("agent-allow.conf") + } + fn seed(&self) -> PathBuf { + self.0.path().join("seed") + } + } + + fn dir(seed: &str) -> Dir { + let dir = Dir(tempfile::tempdir().expect("tempdir")); + std::fs::write(dir.seed(), seed).expect("seed"); + dir + } + + fn sync(d: &Dir, read: Read) -> Result { + sync_with( + read, + &agents(&["atlas"]), + Some(&d.seed()), + "api.anthropic.com cache.nixos.org", + &Files { + allowed: &d.allowed(), + agent_allow: &d.agent_allow(), + }, + ) + } + + fn text(path: &Path) -> String { + std::fs::read_to_string(path).expect("the file is there") + } + + #[test] + fn without_a_swarm_list_the_files_hold_the_seed() { + let d = dir("# a comment\nforge.example\n\n loop.example \n"); + assert_eq!(sync(&d, Read::NotSet).expect("renders"), Source::Seed); + assert_eq!( + text(&d.allowed()), + "# source: seed\nforge.example\nloop.example\n" + ); + assert_eq!(text(&d.agent_allow()), SEED_MARKER); + // A second pass writes nothing. + let ino = std::os::unix::fs::MetadataExt::ino(&std::fs::metadata(d.allowed()).unwrap()); + sync(&d, Read::NotSet).expect("renders"); + assert_eq!( + std::os::unix::fs::MetadataExt::ino(&std::fs::metadata(d.allowed()).unwrap()), + ino + ); + } + + #[test] + fn a_swarm_list_replaces_the_seed_even_when_empty() { + let d = dir("forge.example\n"); + sync(&d, Read::NotSet).expect("seeded"); + assert!(text(&d.allowed()).contains("forge.example"), "control"); + let empty = Read::Current(AllowList::default()); + assert_eq!(sync(&d, empty).expect("renders"), Source::Swarm); + assert_eq!(text(&d.allowed()), SWARM_MARKER); + let named = Read::Current(list(&[("github.com", None), ("pypi.org", Some("atlas"))])); + sync(&d, named).expect("renders"); + assert_eq!(text(&d.allowed()), "# source: swarm\ngithub.com\n"); + assert!(text(&d.agent_allow()).contains("acl agent_allow_atlas proxy_auth atlas")); + } + + #[test] + fn a_swarm_list_that_stops_answering_is_kept_and_a_seed_is_refreshed() { + let d = dir("forge.example\n"); + sync(&d, Read::Current(list(&[("github.com", None)]))).expect("renders"); + let before = (text(&d.allowed()), text(&d.agent_allow())); + assert_eq!(sync(&d, Read::NotSet).expect("renders"), Source::Kept); + assert_eq!((text(&d.allowed()), text(&d.agent_allow())), before); + // Control: a file that never held the swarm's list follows the seed. + let d = dir("forge.example\n"); + std::fs::write(d.allowed(), "old.example\n").expect("a tmpfiles-seeded file"); + assert_eq!(sync(&d, Read::NotSet).expect("renders"), Source::Seed); + assert_eq!(text(&d.allowed()), "# source: seed\nforge.example\n"); + } + + #[test] + fn announce_logs_a_run_of_unset_passes_once() { + let mut announced = None; + announce(&Source::Seed, &mut announced); + assert_eq!(announced, Some("seed")); + announce(&Source::Swarm, &mut announced); + assert_eq!(announced, None, "a list that appears resets the run"); + announce(&Source::Kept, &mut announced); + assert_eq!(announced, Some("kept")); + } + + #[test] + fn the_files_are_group_readable() { + use std::os::unix::fs::PermissionsExt as _; + let d = dir(""); + sync(&d, Read::Current(list(&[("github.com", None)]))).expect("renders"); + for file in [d.allowed(), d.agent_allow()] { + let mode = std::fs::metadata(file).expect("stat").permissions().mode() & 0o777; + assert_eq!(mode, 0o640); + } + } +} diff --git a/hive-c0re/src/workers/squid_users.rs b/hive-c0re/src/workers/squid_users.rs index 8023a3a0..17f1f250 100644 --- a/hive-c0re/src/workers/squid_users.rs +++ b/hive-c0re/src/workers/squid_users.rs @@ -233,7 +233,7 @@ where } /// The file's text, or the empty string when it does not exist. -fn read_or_empty(file: &Path) -> Result { +pub(super) fn read_or_empty(file: &Path) -> Result { match std::fs::read_to_string(file) { Ok(text) => Ok(text), Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(String::new()), @@ -243,7 +243,7 @@ fn read_or_empty(file: &Path) -> Result { /// Write `text` to `file` unless it already holds exactly that. Returns /// whether it wrote. -fn write_if_changed(file: &Path, text: &str) -> Result { +pub(super) fn write_if_changed(file: &Path, text: &str) -> Result { if read_or_empty(file)? == text { return Ok(false); } diff --git a/nix/host-modules/hive-c0re/environment.nix b/nix/host-modules/hive-c0re/environment.nix index e89d5c80..d8ca0eed 100644 --- a/nix/host-modules/hive-c0re/environment.nix +++ b/nix/host-modules/hive-c0re/environment.nix @@ -301,6 +301,13 @@ in # store under the identity above. HIVE_SQUID_USERS_FILE = config.services.hyperhive.squid.usersFile; HIVE_SQUID_UNRESTRICTED_FILE = config.services.hyperhive.squid.unrestrictedFile; + # The swarm's egress allow-list, which `workers::squid_allow` renders into + # the two files; the seed is the list the files hold until the swarm has one, + # and the always-allowed names are those it never repeats. + HIVE_SQUID_ALLOWED_FILE = config.services.hyperhive.squid.allowedFile; + HIVE_SQUID_AGENT_ALLOW_FILE = config.services.hyperhive.squid.agentAllowFile; + HIVE_SQUID_ALLOWED_SEED_FILE = config.services.hyperhive.squid.allowedSeedFile; + HIVE_SQUID_ALWAYS_ALLOWED = lib.concatStringsSep " " config.services.hyperhive.squid.alwaysAllowed; # The egress proxy facts every container is handed (`HiveFacts` in # `hive-priv-sock`). They sit with the users file above because that is what # makes squid's credential check and per-agent policy real on this host: an diff --git a/nix/host-modules/hive-squid.nix b/nix/host-modules/hive-squid.nix index ca727f26..a6a2dcd4 100644 --- a/nix/host-modules/hive-squid.nix +++ b/nix/host-modules/hive-squid.nix @@ -4,7 +4,10 @@ # must carry a per-agent proxy credential, checked against the users file # hive-c0re renders. A credential's agent either has the swarm's # `unrestricted_internet` flag, and may then reach any destination the deny -# rules leave, or reaches only the listed names. An agent sends its credential +# rules leave, or reaches only the listed names: the base list, the allow-list +# the swarm keeps (rendered by hive-c0re beside the users file, with this +# host's `allowedDomains` as the list until the swarm has one) and the names +# the swarm lists for that agent alone. An agent sends its credential # once it has fetched it (../agent-modules/egress-proxy.nix); the exporter, # journal forwarding and the self-check below show the proxy works. { @@ -26,6 +29,16 @@ let allowed = lib.unique (cfg.allowedDomains ++ [ cfg.selfcheck.canary ]); base = lib.unique cfg.baseAllowedDomains; + # The names squid allows every authenticated user whatever the swarm lists. + # hive-c0re skips an allow-list entry that one of them already covers. + alwaysAllowed = base ++ [ cfg.selfcheck.canary ]; + # What `allowedFile` holds until the swarm has an allow-list: this hive's + # own `allowedDomains`, one per line. The canary is in the configuration. + seedFile = pkgs.writeText "squid-allowed-seed" ( + lib.concatMapStrings (d: d + "\n") ( + lib.unique (lib.subtractLists [ cfg.selfcheck.canary ] cfg.allowedDomains) + ) + ); # The squid user the self-check logs in as: `swarm_secret_client::proxy::SELFCHECK_USER`. selfcheckUser = "selfcheck.squid"; @@ -65,8 +78,10 @@ let # denies come first because `dstdomain` on an IP literal does a reverse # lookup. The credential check sits right before the first allow, so only a # request that would otherwise be let through gets the 407 challenge. Every - # user is then let through to the base names, an unrestricted user to any other - # destination on a safe port, and every other user to the listed names. The `dst` denies cost an + # user is then let through to the base names, a user the swarm lists names + # for to those names (the include, whose rules hive-c0re generates and whose + # file may be empty), an unrestricted user to any other destination on a + # safe port, and every other user to the listed names. The `dst` denies cost an # unlisted name a lookup on the host's resolver before it is refused. squidConf = '' http_port ${listen} @@ -107,9 +122,8 @@ let acl ip_v4_host url_regex ^([a-z]+://)?[0-9.]+(:[0-9]+)?(/|$) acl ip_v6_host url_regex ^([a-z]+://)?\[[0-9a-fA-F:.]+\](:[0-9]+)?(/|$) ${lib.optionalString (base != [ ]) "acl base_dst dstdomain ${lib.concatStringsSep " " base}"} - ${lib.optionalString ( - allowed != [ ] - ) "acl allowed_dst dstdomain ${lib.concatStringsSep " " allowed}"} + acl allowed_dst dstdomain ${cfg.selfcheck.canary} + acl allowed_dst dstdomain "${cfg.allowedFile}" # Answered by squid itself, never forwarded, and addressed to the bridge IP # on a port outside Safe_ports, so it precedes the destination rules. @@ -126,12 +140,30 @@ let http_access deny to_private http_access deny !authed ${lib.optionalString (base != [ ]) "http_access allow base_dst"} + include ${cfg.agentAllowFile} http_access allow unrestricted - ${lib.optionalString (allowed != [ ]) "http_access deny !allowed_dst"} - ${lib.optionalString (allowed != [ ]) "http_access allow allowed_dst"} + http_access deny !allowed_dst + http_access allow allowed_dst http_access deny all ''; + # `squid -k parse` on the configuration with the include pointing at an empty + # file. The nixpkgs module runs the same parse on the real text, which + # cannot open the include in the build sandbox. + squidConfCheck = + pkgs.runCommand "squid-conf-check" + { + nativeBuildInputs = [ config.services.squid.package ]; + } + '' + squid -k parse -f ${ + pkgs.writeText "squid.conf" ( + builtins.replaceStrings [ cfg.agentAllowFile ] [ "/dev/null" ] squidConf + ) + } + touch $out + ''; + sandbox = { NoNewPrivileges = true; ProtectSystem = "strict"; @@ -222,8 +254,14 @@ in "crates.io" ]; description = '' - Hostnames squid lets a client reach, by CONNECT or plain HTTP. Each - entry matches that exact name. Everything else gets a 403, except + Hostnames squid lets a client reach, by CONNECT or plain HTTP, until the + swarm has an allow-list of its own: the list the swarm's controller + keeps (`/api/egress/allowed-domains`) replaces this one as soon as it + holds an entry or has had one, and this option is the list again only + while the controller has none, for instance because it predates the + feature. hive-c0re writes whichever list applies to `allowedFile`. + Each entry matches that exact name, or that name and its subdomains + with a leading dot. Everything else gets a 403, except for an unrestricted user. The swarm's own service names are not listed: they resolve to the bridge or a private address, which the destination rules refuse, and agents reach them directly. The @@ -238,11 +276,7 @@ in baseAllowedDomains = lib.mkOption { type = lib.types.listOf lib.types.str; - default = [ - "api.anthropic.com" - "claude.ai" - "platform.claude.com" - ]; + default = import ./lib/squid-base-domains.nix; example = [ "api.anthropic.com" "claude.ai" @@ -261,7 +295,9 @@ in environment, which this host does not read. Swarm-internal names are not listed, since agents reach them directly. The destination denies still apply to every entry, and the list is separate from - `allowedDomains`. + `allowedDomains`. A name the swarm's allow-list holds that this list + already covers is left out of the rendered files, and the controller + refuses it as an entry when it is among the defaults. ''; }; @@ -289,12 +325,66 @@ in `unrestricted_internet` flag (set unless the swarm records it off), and the self-check's unrestricted user. hive-c0re renders it beside `usersFile`, and squid is reloaded when it changes. Every other user - reaches only `allowedDomains`. An empty file, which is what exists + reaches only the base list, the allow-list and its own names. An empty file, which is what exists until hive-c0re writes one, makes every user a restricted one. Owned by `hive-core` and readable by the `squid` group. ''; }; + allowedFile = lib.mkOption { + type = lib.types.path; + default = "${builtins.dirOf cfg.usersFile}/allowed"; + readOnly = true; + defaultText = lib.literalMD "`allowed`, in the directory of `usersFile`"; + description = '' + The names squid lets every restricted user reach, one per line, read as + part of its allow ACL beside the self-check canary. It starts as a copy + of `allowedDomains`. hive-c0re rewrites it every pass: with the swarm's + swarm-wide entries once the controller has a list, otherwise with + `allowedDomains`. Squid is reloaded when it changes. Owned by + `hive-core` and readable by the `squid` group. + ''; + }; + + agentAllowFile = lib.mkOption { + type = lib.types.path; + default = "${builtins.dirOf cfg.usersFile}/agent-allow.conf"; + readOnly = true; + defaultText = lib.literalMD "`agent-allow.conf`, in the directory of `usersFile`"; + description = '' + Squid configuration that hive-c0re generates and the main configuration + includes after every destination deny and `deny !authed`, where the + base list is allowed: for each agent of this hive the swarm lists + names for, an ACL for its user, an ACL for its names and the + `http_access allow` joining the two. Empty until the controller has a + list. Squid is reloaded when it changes. Owned by `hive-core` and + readable by the `squid` group. + ''; + }; + + allowedSeedFile = lib.mkOption { + type = lib.types.path; + default = seedFile; + readOnly = true; + defaultText = lib.literalMD "`allowedDomains`, one per line, minus the self-check canary"; + description = '' + The names `allowedFile` starts as and hive-c0re writes to it again while + the swarm has no allow-list. Read-only. + ''; + }; + + alwaysAllowed = lib.mkOption { + type = lib.types.listOf lib.types.str; + default = alwaysAllowed; + readOnly = true; + defaultText = lib.literalMD "`baseAllowedDomains` and the self-check canary"; + description = '' + The names squid allows every authenticated user whatever the swarm + lists. hive-c0re leaves a swarm allow-list entry out of the rendered + files when one of these covers it. Read-only. + ''; + }; + selfcheck.credentialName = lib.mkOption { type = lib.types.str; default = "hive-squid.selfcheck"; @@ -348,31 +438,45 @@ in services.squid = { enable = true; configText = squidConf; + # The module's build-time parse fails on the `include` of a file only the + # running host has; `squidConfCheck` parses the same text without it. + validateConfig = false; }; + system.extraDependencies = [ squidConfCheck ]; + # The directory is setgid so the file hive-c0re renames into it is the # squid group's. The users file exists before squid starts, empty: the # helper exits on a missing file and squid then rejects every login. The # unrestricted list is created the same way, though squid treats a missing - # one as empty. + # one as empty. The allow-list file starts as a copy of `allowedDomains` + # and an existing one is left alone, so the list hive-c0re wrote survives a + # switch. The include is empty until hive-c0re has a list: squid does not + # start on a missing one. systemd.tmpfiles.rules = [ "d ${usersDir} 2750 hive-core squid - -" "f ${cfg.usersFile} 0640 hive-core squid - -" "f ${cfg.unrestrictedFile} 0640 hive-core squid - -" + "C ${cfg.allowedFile} 0640 hive-core squid - ${seedFile}" + "f ${cfg.agentAllowFile} 0640 hive-core squid - -" ]; - # Squid reads the unrestricted list when it parses its configuration, not - # when it changes. hive-c0re writes the list only when its contents change, - # so each change is one reload. Nothing happens to a squid that is not - # running; it reads the list when it starts. + # Squid reads the unrestricted list, the allow-list and the per-agent include + # when it parses its configuration, not when they change. hive-c0re writes + # each only when its contents change, so each change is one reload. Nothing + # happens to a squid that is not running; it reads them when it starts. systemd.paths.squid-unrestricted-reload = { - description = "Watch squid's unrestricted-users list"; + description = "Watch the lists hive-c0re renders for squid"; wantedBy = [ "paths.target" ]; - pathConfig.PathChanged = cfg.unrestrictedFile; + pathConfig.PathChanged = [ + cfg.unrestrictedFile + cfg.allowedFile + cfg.agentAllowFile + ]; }; systemd.services.squid-unrestricted-reload = { - description = "Reload squid to read its unrestricted-users list"; + description = "Reload squid to read the lists hive-c0re renders"; # Several list writes in a few seconds must not trip the start limit: a # tripped limit also fails the path unit, and later writes then reload nothing. startLimitIntervalSec = 0; diff --git a/nix/host-modules/lib/squid-base-domains.nix b/nix/host-modules/lib/squid-base-domains.nix new file mode 100644 index 00000000..b63e4f5b --- /dev/null +++ b/nix/host-modules/lib/squid-base-domains.nix @@ -0,0 +1,9 @@ +# The default of `services.hyperhive.squid.baseAllowedDomains`, and the names +# swarm-controller refuses as allow-list entries because squid already allows +# them to every authenticated user: the hosts Claude Code requires +# (). +[ + "api.anthropic.com" + "claude.ai" + "platform.claude.com" +] diff --git a/nix/host-modules/swarm-controller.nix b/nix/host-modules/swarm-controller.nix index c9933720..5890046e 100644 --- a/nix/host-modules/swarm-controller.nix +++ b/nix/host-modules/swarm-controller.nix @@ -864,6 +864,14 @@ in # token is in the store, written by `swarm-bao-grafana-controller-token` # on Grafana's host. SWARM_CONTROLLER_GRAFANA_URL = "https://${swarmCfg.grafana.domain}"; + # The names squid allows every authenticated agent by default. The + # allow-list route refuses them as entries + # (swarm-controller/src/egress_allow.rs). A hive that overrides + # `squid.baseAllowedDomains` is not reflected here; its hive-c0re + # skips the entries its own list covers. + SWARM_CONTROLLER_SQUID_BASE_DOMAINS = lib.concatStringsSep " " ( + import ./lib/squid-base-domains.nix + ); } // queueEnv // forgeEnv diff --git a/nix/module-eval/bao-controller.nix b/nix/module-eval/bao-controller.nix index 3d00c2ab..f32fbf8a 100644 --- a/nix/module-eval/bao-controller.nix +++ b/nix/module-eval/bao-controller.nix @@ -249,6 +249,20 @@ let controllerNoStore ]; } + { + # The allow-list route refuses the names squid already allows everyone. + # The list the controller is handed is the hive module's default, from + # one file; the second clause is the control that the default is not empty. + name = "the controller is handed the base list squid defaults to"; + ok = + let + base = baoControllerHere.services.hyperhive.squid.baseAllowedDomains; + in + base != [ ] + && + baoControllerHere.systemd.services.swarm-controller.environment.SWARM_CONTROLLER_SQUID_BASE_DOMAINS + == lib.concatStringsSep " " base; + } { # A hive's cert-auth role carries the authority by value, so the daemon # has to be handed the file rather than a path into the store's own diff --git a/nix/module-eval/hive-otel.nix b/nix/module-eval/hive-otel.nix index b408268f..cf378a95 100644 --- a/nix/module-eval/hive-otel.nix +++ b/nix/module-eval/hive-otel.nix @@ -47,6 +47,12 @@ let hiveOtelAllLocal = hiveWithController { deploy.bao.enable = true; deploy.authelia.enable = true; + # The seed `allowedFile` starts as; the canary is listed too, to show it + # is not written twice. + squid.allowedDomains = [ + "github.com" + "cache.nixos.org" + ]; }; # Neither a leaf nor a secret file: the one shape the collector has no @@ -500,12 +506,18 @@ let base = at "http_access allow base_dst"; unrestricted = at "http_access allow unrestricted"; authed = at "http_access deny !authed"; - firstAllow = lib.min base (lib.min unrestricted allow); + # The generated per-agent `http_access allow` rules sit in this file. + agentRules = at "include ${hiveOtel.services.hyperhive.squid.agentAllowFile}"; + firstAllow = lib.min agentRules (lib.min base (lib.min unrestricted allow)); in allow != null && base != null && unrestricted != null && authed != null + && agentRules != null + && authed < agentRules + && base < agentRules + && agentRules < unrestricted && at "http_access deny to_localhost" != null && at "http_access deny to_linklocal" != null && at "http_access deny to_bridge" != null @@ -602,13 +614,55 @@ let path = hiveOtel.systemd.paths.squid-unrestricted-reload; svc = hiveOtel.systemd.services.squid-unrestricted-reload.serviceConfig; in - path.pathConfig.PathChanged == list + path.pathConfig.PathChanged == [ + list + hiveOtel.services.hyperhive.squid.allowedFile + hiveOtel.services.hyperhive.squid.agentAllowFile + ] && hiveOtel.systemd.services.squid-unrestricted-reload.startLimitIntervalSec == 0 && builtins.elem "paths.target" path.wantedBy && lib.hasSuffix "systemctl try-reload-or-restart squid.service" svc.ExecStart && builtins.elem "f ${list} 0640 hive-core squid - -" rules && hiveOtelAllLocal.systemd.services.hive-c0re.environment.HIVE_SQUID_UNRESTRICTED_FILE == list; } + { + # The swarm's allow-list reaches squid as a file of names, which starts + # as this hive's own `allowedDomains` (the canary left out, since the + # configuration names it), and as an include that holds the per-agent + # rules. Control: the seed holds the listed name, and the configuration + # no longer names it, so the absence of the canary and of the list from + # the seed and the configuration is not a lookup of nothing. + name = "squid reads the swarm's allow-list from a file seeded with the hive's own names"; + ok = + let + sq = hiveOtelAllLocal.services.hyperhive.squid; + lines = lib.splitString "\n" hiveOtelAllLocal.services.squid.configText; + rules = hiveOtelAllLocal.systemd.tmpfiles.rules; + env = hiveOtelAllLocal.systemd.services.hive-c0re.environment; + in + builtins.elem ''acl allowed_dst dstdomain "${sq.allowedFile}"'' lines + && builtins.elem "acl allowed_dst dstdomain ${sq.selfcheck.canary}" lines + && !(lib.hasInfix "github.com" hiveOtelAllLocal.services.squid.configText) + && sq.allowedFile == "/var/lib/hive-squid/allowed" + && sq.agentAllowFile == "/var/lib/hive-squid/agent-allow.conf" + && sq.allowedSeedFile.text == "github.com\n" + && builtins.elem "C ${sq.allowedFile} 0640 hive-core squid - ${sq.allowedSeedFile}" rules + && builtins.elem "f ${sq.agentAllowFile} 0640 hive-core squid - -" rules + && env.HIVE_SQUID_ALLOWED_FILE == sq.allowedFile + && env.HIVE_SQUID_AGENT_ALLOW_FILE == sq.agentAllowFile + && env.HIVE_SQUID_ALLOWED_SEED_FILE == toString sq.allowedSeedFile + && + env.HIVE_SQUID_ALWAYS_ALLOWED == "api.anthropic.com claude.ai platform.claude.com cache.nixos.org"; + } + { + # The nixpkgs module parses the configuration at build time, which cannot + # open the include; the parse moves to a check on the same text, so a + # broken configuration still fails the host build. + name = "squid's configuration is still parsed at build time, without the include"; + ok = + !hiveOtel.services.squid.validateConfig + && lib.any (d: (d.name or "") == "squid-conf-check") hiveOtel.system.extraDependencies; + } { # The helper is the NCSA one, on the file hive-c0re renders. The file # exists, empty, before squid starts (the helper exits on a missing one), diff --git a/nix/vm-tests/hive-squid.nix b/nix/vm-tests/hive-squid.nix index 87acdd4c..37c4d3c1 100644 --- a/nix/vm-tests/hive-squid.nix +++ b/nix/vm-tests/hive-squid.nix @@ -152,6 +152,8 @@ pkgs.testers.runNixOSTest { "cache.t.local" "api.t.local" "other.t.local" + "swarm.t.local" + "agentonly.t.local" "deny-canary.invalid" ]; "127.0.0.1" = [ "loop.t.local" ]; @@ -231,6 +233,7 @@ pkgs.testers.runNixOSTest { testScript = '' import re import secrets + import shlex import time # Drawn fresh each run. The users file holds only their hashes, as @@ -248,6 +251,8 @@ pkgs.testers.runNixOSTest { RUTH = "ruth:" + PW["ruth"] USERS = "/var/lib/hive-squid/users" LIST = "/var/lib/hive-squid/unrestricted" + ALLOWED = "/var/lib/hive-squid/allowed" + AGENT_ALLOW = "/var/lib/hive-squid/agent-allow.conf" UNRESTRICTED = ["uma", "selfcheck-unrestricted.squid"] start_all() @@ -289,6 +294,16 @@ pkgs.testers.runNixOSTest { f"&& chmod 640 {LIST}.new && mv {LIST}.new {LIST}" ) + def write_allow(allowed, agent_conf): + # What hive-c0re does for its two files: written aside and renamed into + # place, in squid's group. Squid reads them on a reload, which + # squid-unrestricted-reload does when either changes. + for path, body in ((ALLOWED, allowed), (AGENT_ALLOW, agent_conf)): + proxy.succeed( + f"printf '%s' {shlex.quote(body)} > {path}.new && chown hive-core:squid {path}.new " + f"&& chmod 640 {path}.new && mv {path}.new {path}" + ) + def wait_status(host, cred, want): # The reload that follows a list write is asynchronous. got = None @@ -310,6 +325,11 @@ pkgs.testers.runNixOSTest { assert proxy.succeed("stat -c %a /var/lib/hive-squid").strip() == "2750" assert proxy.succeed(f"stat -c %s {USERS}").strip() == "0" assert proxy.succeed(f"stat -c %s {LIST}").strip() == "0" + assert proxy.succeed(f"stat -c '%U:%G %a' {ALLOWED}").strip() == "hive-core:squid 640" + assert proxy.succeed(f"stat -c '%U:%G %a' {AGENT_ALLOW}").strip() == "hive-core:squid 640" + assert proxy.succeed(f"stat -c %s {AGENT_ALLOW}").strip() == "0" + # The allow-list file starts as the hive's own allowedDomains. + proxy.succeed(f"grep -Fx forge.t.local {ALLOWED}") assert connect("forge.t.local", ATLAS) == "407", "no such user" assert connect("other.t.local", UMA) == "407", "no such user, unlisted name" @@ -500,6 +520,31 @@ pkgs.testers.runNixOSTest { wait_status("other.t.local", RUTH, "403") assert connect("other.t.local", UMA) == "200", "control: uma is still listed" + with subtest("the swarm's allow-list replaces the hive's names, opens one to every restricted user, and a per-agent rule opens one to that agent alone"): + seeded = proxy.succeed(f"cat {ALLOWED}") + assert connect("swarm.t.local", RUTH) == "403", "control: not allowed yet" + assert connect("agentonly.t.local", ATLAS) == "403", "control: not allowed yet" + assert connect("forge.t.local", RUTH) == "200", "control: the hive's own name" + write_allow( + "# source: swarm\nswarm.t.local\n", + "# source: swarm\n" + "acl agent_allow_atlas proxy_auth atlas\n" + "acl agent_allow_atlas_dst dstdomain agentonly.t.local priv10.t.local\n" + "http_access allow agent_allow_atlas agent_allow_atlas_dst\n", + ) + wait_status("swarm.t.local", RUTH, "200") + wait_status("agentonly.t.local", ATLAS, "200") + assert connect("swarm.t.local", ATLAS) == "200", "swarm-wide, another user" + assert connect("agentonly.t.local", RUTH) == "403", "the per-agent name, another restricted user" + assert connect("agentonly.t.local") == "407", "no credential" + assert connect("forge.t.local", RUTH) == "403", "the hive's own name is replaced" + assert connect("priv10.t.local", ATLAS) == "403", "a per-agent name that resolves privately" + assert connect("agentonly.t.local", UMA) == "200", "control: unrestricted reaches it anyway" + write_allow(seeded, "") + wait_status("swarm.t.local", RUTH, "403") + wait_status("agentonly.t.local", ATLAS, "403") + assert connect("forge.t.local", RUTH) == "200", "the hive's own name is back" + with subtest("squid starts on a missing list, treats every user as restricted, and recovers when it appears"): proxy.succeed(f"rm {LIST}") proxy.succeed("systemctl restart squid.service") diff --git a/swarm-controller-api/src/egress.rs b/swarm-controller-api/src/egress.rs new file mode 100644 index 00000000..70258053 --- /dev/null +++ b/swarm-controller-api/src/egress.rs @@ -0,0 +1,56 @@ +//! `/api/egress/allowed-domains` — the destinations the swarm lets agents +//! reach through their hive's egress proxy. + +use serde::{Deserialize, Serialize}; + +/// One allowed destination and who may reach it. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[cfg_attr(feature = "schema", derive(utoipa::ToSchema))] +#[serde(deny_unknown_fields)] +pub struct AllowedDomain { + /// A DNS name of at least two labels, such as `github.com`, stored + /// lower-cased. One leading dot, as in `.githubusercontent.com`, also + /// allows every subdomain. IP addresses, wildcards, ports, paths and the + /// proxy's own always-allowed names are refused. + #[cfg_attr(feature = "schema", schema(example = "github.com"))] + pub domain: String, + /// The agent this entry is for. Absent means every agent in the swarm. + #[serde(default, skip_serializing_if = "Option::is_none")] + #[cfg_attr(feature = "schema", schema(example = "atlas"))] + pub agent: Option, +} + +/// The swarm's allow-list: every entry, swarm-wide and per agent, sorted. +#[derive(Debug, Default, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[cfg_attr(feature = "schema", derive(utoipa::ToSchema))] +pub struct AllowedDomains { + pub entries: Vec, +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn an_entry_leaves_out_an_unset_agent_and_refuses_unknown_fields() { + let swarm_wide = AllowedDomain { + domain: "github.com".to_owned(), + agent: None, + }; + let wire = r#"{"domain":"github.com"}"#; + assert_eq!(serde_json::to_string(&swarm_wide).unwrap(), wire); + assert_eq!( + serde_json::from_str::(wire).unwrap(), + swarm_wide + ); + let scoped = r#"{"domain":"crates.io","agent":"atlas"}"#; + assert_eq!( + serde_json::from_str::(scoped) + .unwrap() + .agent + .as_deref(), + Some("atlas") + ); + assert!(serde_json::from_str::(r#"{"domain":"a.io","hive":"x"}"#).is_err()); + } +} diff --git a/swarm-controller-api/src/lib.rs b/swarm-controller-api/src/lib.rs index d62e26e0..b5585c45 100644 --- a/swarm-controller-api/src/lib.rs +++ b/swarm-controller-api/src/lib.rs @@ -6,6 +6,7 @@ pub mod agents; pub mod agents_md; +pub mod egress; pub mod forge; pub mod inference; pub mod limits; diff --git a/swarm-controller/README.md b/swarm-controller/README.md index 252e30b3..545ca62c 100644 --- a/swarm-controller/README.md +++ b/swarm-controller/README.md @@ -62,6 +62,13 @@ The swarm's control plane. The swarm UI and `swarmctl` are its clients. (`GET`/`PUT`/`DELETE /api/agents/{name}/limits`); the hive running the agent reads it, and each cap replaces the hive's own answer for that field → [`docs/scheduler/coordinator.md`](../docs/scheduler/coordinator.md#container-resource-limits). +- **Egress allow-list** — the destinations agents may reach through their + hive's egress proxy, each for every agent or for one, as one value in the + `egress-allow` KV bucket, which this daemon alone writes + (`GET`/`POST`/`DELETE /api/egress/allowed-domains`); every hive reads it and + renders the proxy's allow-list files, and it replaces the list the hive's own + configuration names → + [`docs/networking/network.md`](../docs/networking/network.md#egress-proxy-squid). - **Agent instructions** — each agent's `AGENTS.md` as its harness last published it into the `agents-md` KV bucket (`GET /api/agents/{name}/agents-md`), and edits to it as git diffs, which diff --git a/swarm-controller/src/egress_allow.rs b/swarm-controller/src/egress_allow.rs new file mode 100644 index 00000000..5e927a6e --- /dev/null +++ b/swarm-controller/src/egress_allow.rs @@ -0,0 +1,557 @@ +//! `/api/egress/allowed-domains` — the destinations the swarm lets agents +//! reach through their hive's egress proxy. +//! +//! The whole list is one value under one key of `egress-allow` +//! (`swarm_wire::egress_allow`), which this daemon alone writes. Every hive +//! reads it and renders the entries for the agents it runs: a swarm-wide entry +//! joins the proxy's allow list for everyone, an entry with an agent is that +//! agent's alone. Once the key exists it replaces the list a hive's own nix +//! configuration names, so an empty list is a list. +//! +//! Each name goes into the proxy's configuration, so it is checked here with +//! `swarm_wire::egress_allow::normalize_domain` (the hive checks again) and +//! refused when the proxy already allows it to everyone. The names the proxy +//! always allows are the ones in [`ENV_BASE_DOMAINS`]. + +use axum::Json; +use axum::extract::State; +use axum::http::StatusCode; +use problem_details::ProblemDetails; +use swarm_controller_api::egress::{AllowedDomain, AllowedDomains}; +use swarm_queue_client::egress_allow; +use swarm_wire::egress_allow::{AllowList, Entry, KEY, MAX_ENTRIES, covered_by, normalize_domain}; + +use crate::routes::{AppState, error_problem, wanted_error_status}; +use crate::schedules::{live, queue, read_or_default, refused, write_revision}; + +/// How many times a losing writer re-reads and re-applies, as in +/// `wanted::MAX_ATTEMPTS`. +const MAX_ATTEMPTS: usize = 5; + +/// Set from the hive module's `baseAllowedDomains` default by +/// `nix/host-modules/swarm-controller.nix`: the names, separated by spaces, +/// that every hive's proxy lets every authenticated agent reach. Unset means +/// none are known and nothing is refused on that ground; a hive still skips +/// the entries its own base list covers. +pub(crate) const ENV_BASE_DOMAINS: &str = "SWARM_CONTROLLER_SQUID_BASE_DOMAINS"; + +fn base_domains() -> Vec { + std::env::var(ENV_BASE_DOMAINS) + .unwrap_or_default() + .split_whitespace() + .map(str::to_ascii_lowercase) + .collect() +} + +/// The entry to store for `req`, or why it is refused. +pub(crate) fn checked(req: &AllowedDomain, base: &[String]) -> Result { + let domain = normalize_domain(&req.domain)?; + if covered_by(&domain, base) { + return Err(format!( + "{domain} is already allowed for every agent by the proxy's base list" + )); + } + let agent = req + .agent + .as_deref() + .map(|a| { + hive_types::Ident::parse(a) + .map(hive_types::Ident::into_string) + .map_err(|reason| format!("agent {a:?}: {reason}")) + }) + .transpose()?; + Ok(Entry { domain, agent }) +} + +/// What adding an entry to a list came to. +enum Added { + /// The entry was there: nothing to write. + Already(AllowList), + /// The list with the entry in it. + Now(AllowList), +} + +/// `list` with `entry` added, or why it can't take another entry. +fn added(mut list: AllowList, entry: &Entry) -> Result { + if list.entries.contains(entry) { + return Ok(Added::Already(list)); + } + if list.entries.len() >= MAX_ENTRIES { + return Err(format!( + "the list holds {MAX_ENTRIES} entries, which is the most it takes" + )); + } + list.entries.push(entry.clone()); + list.entries.sort(); + Ok(Added::Now(list)) +} + +fn view(list: AllowList) -> AllowedDomains { + let mut entries = list.entries; + entries.sort(); + AllowedDomains { + entries: entries + .into_iter() + .map(|e| AllowedDomain { + domain: e.domain, + agent: e.agent, + }) + .collect(), + } +} + +fn queue_problem(e: &anyhow::Error) -> ProblemDetails { + tracing::warn!(error = %format!("{e:#}"), "egress allow-list: queue failure"); + error_problem(wanted_error_status(e), &format!("{e:#}")) +} + +async fn open_or_create( + client: &async_nats::Client, +) -> Result { + egress_allow::open_or_create(client) + .await + .map_err(|e| queue_problem(&anyhow::Error::new(e))) +} + +#[utoipa::path( + get, + path = "/api/egress/allowed-domains", + responses( + (status = 200, description = "every entry, swarm-wide and per agent, sorted; empty \ + when the swarm has not set a list, in which case each hive keeps its own", + body = AllowedDomains), + (status = 503, description = "no swarm queue is configured on this host, or it is \ + not connected (problem+json)", body = String), + (status = 500, description = "the list could not be read (problem+json)", + body = String), + ), + tag = "egress" +)] +pub(crate) async fn get_allowed_domains( + State(state): State, +) -> Result, ProblemDetails> { + let client = queue(&state).map_err(refused)?; + let Some(store) = egress_allow::open(&client) + .await + .map_err(|e| queue_problem(&anyhow::Error::new(e)))? + else { + return Ok(Json(AllowedDomains::default())); + }; + let (current, _, _) = read_or_default::(&store, KEY) + .await + .map_err(|e| queue_problem(&e))?; + Ok(Json(view(current))) +} + +/// Allow one destination, for every agent or for one. +/// +/// Adding an entry that is already there changes nothing. The first entry +/// makes the swarm's list replace each hive's own, so a name the hive's nix +/// configuration allows and this list lacks stops being allowed once hives read +/// it. +#[utoipa::path( + post, + path = "/api/egress/allowed-domains", + request_body = AllowedDomain, + responses( + (status = 200, description = "the whole list after the add", body = AllowedDomains), + (status = 400, description = "the domain is not a DNS name (an IP address, a \ + wildcard, a single label and a port are refused), the proxy already allows \ + it to everyone, the agent is not an identifier or not declared on any hive, \ + or the list is full (problem+json)", body = String), + (status = 422, description = "an unknown field"), + (status = 503, description = "no swarm queue is configured on this host, or it is \ + not connected (problem+json)", body = String), + (status = 500, description = "the list could not be written (problem+json)", + body = String), + ), + tag = "egress" +)] +pub(crate) async fn add_allowed_domain( + State(state): State, + Json(req): Json, +) -> Result, ProblemDetails> { + let entry = checked(&req, &base_domains()) + .map_err(|why| error_problem(StatusCode::BAD_REQUEST, &why))?; + let client = queue(&state).map_err(refused)?; + if let Some(agent) = &entry.agent + && !live(&state).await?.contains(agent) + { + return Err(error_problem( + StatusCode::BAD_REQUEST, + &format!("no hive's wanted state declares an agent {agent:?} that is not destroyed"), + )); + } + let store = open_or_create(&client).await?; + for _ in 0..MAX_ATTEMPTS { + let (list, revision, _) = read_or_default::(&store, KEY) + .await + .map_err(|e| queue_problem(&e))?; + let list = match added(list, &entry) { + Ok(Added::Already(list)) => return Ok(Json(view(list))), + Ok(Added::Now(list)) => list, + Err(why) => return Err(error_problem(StatusCode::BAD_REQUEST, &why)), + }; + if write_revision(&store, KEY, &list, revision) + .await + .map_err(|e| queue_problem(&e))? + .is_some() + { + tracing::info!(domain = %entry.domain, agent = ?entry.agent, "allowed a domain"); + return Ok(Json(view(list))); + } + } + Err(queue_problem(&anyhow::anyhow!( + "gave up adding {} after {MAX_ATTEMPTS} conflicting writes", + entry.domain + ))) +} + +/// Stop allowing one destination for the scope it was added with. +/// +/// Removing an entry that is not there changes nothing. Removing the last one +/// leaves an empty list, which still replaces each hive's own. +#[utoipa::path( + delete, + path = "/api/egress/allowed-domains", + request_body = AllowedDomain, + responses( + (status = 200, description = "the whole list after the removal", body = AllowedDomains), + (status = 400, description = "the domain is not a DNS name, or the agent is not an \ + identifier (problem+json)", body = String), + (status = 422, description = "an unknown field"), + (status = 503, description = "no swarm queue is configured on this host, or it is \ + not connected (problem+json)", body = String), + (status = 500, description = "the list could not be written (problem+json)", + body = String), + ), + tag = "egress" +)] +pub(crate) async fn remove_allowed_domain( + State(state): State, + Json(req): Json, +) -> Result, ProblemDetails> { + // The base list is not consulted: an entry stored before a name joined it + // can still be removed. + let entry = checked(&req, &[]).map_err(|why| error_problem(StatusCode::BAD_REQUEST, &why))?; + let client = queue(&state).map_err(refused)?; + let store = open_or_create(&client).await?; + for _ in 0..MAX_ATTEMPTS { + let (mut list, revision, _) = read_or_default::(&store, KEY) + .await + .map_err(|e| queue_problem(&e))?; + if !list.entries.contains(&entry) { + return Ok(Json(view(list))); + } + list.entries.retain(|e| *e != entry); + if write_revision(&store, KEY, &list, revision) + .await + .map_err(|e| queue_problem(&e))? + .is_some() + { + tracing::info!(domain = %entry.domain, agent = ?entry.agent, "stopped allowing a domain"); + return Ok(Json(view(list))); + } + } + Err(queue_problem(&anyhow::anyhow!( + "gave up removing {} after {MAX_ATTEMPTS} conflicting writes", + entry.domain + ))) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::agent_limits::tests::{declared, state}; + use crate::test_nats::TestNats; + + fn entry(domain: &str, agent: Option<&str>) -> AllowedDomain { + AllowedDomain { + domain: domain.to_owned(), + agent: agent.map(str::to_owned), + } + } + + async fn add( + state: &AppState, + domain: &str, + agent: Option<&str>, + ) -> Result { + add_allowed_domain(State(state.clone()), Json(entry(domain, agent))) + .await + .map(|j| j.0) + } + + async fn remove( + state: &AppState, + domain: &str, + agent: Option<&str>, + ) -> Result { + remove_allowed_domain(State(state.clone()), Json(entry(domain, agent))) + .await + .map(|j| j.0) + } + + async fn listed(state: &AppState) -> Vec<(String, Option)> { + get_allowed_domains(State(state.clone())) + .await + .expect("the list is read") + .0 + .entries + .into_iter() + .map(|e| (e.domain, e.agent)) + .collect() + } + + /// The raw value a hive reads, or `None` for no key. + async fn published(client: &async_nats::Client) -> Option { + egress_allow::open_or_create(client) + .await + .expect("bucket") + .get(KEY) + .await + .expect("read") + .map(|v| String::from_utf8(v.to_vec()).expect("utf-8")) + } + + fn base() -> Vec { + vec!["api.anthropic.com".to_owned(), ".claude.ai".to_owned()] + } + + #[test] + fn a_name_is_stored_normalised_and_an_ip_literal_is_refused() { + let stored = checked(&entry(" GitHub.com ", Some("atlas")), &base()).expect("a name"); + assert_eq!(stored.domain, "github.com"); + assert_eq!(stored.agent.as_deref(), Some("atlas")); + let dotted = checked(&entry(".githubusercontent.com", None), &base()).expect("dot form"); + assert_eq!(dotted.domain, ".githubusercontent.com"); + assert_eq!(dotted.agent, None); + + for bad in [ + "1.2.3.4", + "127.0.0.1", + "[::1]", + "::1", + "*.example.com", + "localhost", + "example.com:443", + "example.com\nhttp_access allow all", + ] { + assert!( + checked(&entry(bad, None), &base()).is_err(), + "{bad:?} was accepted" + ); + } + for agent in ["", "At Las", "a/b", "../argus"] { + assert!( + checked(&entry("github.com", Some(agent)), &base()).is_err(), + "agent {agent:?} was accepted" + ); + } + } + + #[test] + fn a_name_the_base_list_already_allows_is_refused() { + for covered in [ + "api.anthropic.com", + "claude.ai", + "a.claude.ai", + ".claude.ai", + ] { + let why = checked(&entry(covered, None), &base()).expect_err(covered); + assert!(why.contains("base list"), "{why}"); + } + // Control: the same names pass when the base list is unknown, and names + // next to the base ones are not refused. + assert!(checked(&entry("claude.ai", None), &[]).is_ok()); + for fresh in ["x.api.anthropic.com", ".anthropic.com", "notclaude.ai"] { + assert!(checked(&entry(fresh, None), &base()).is_ok(), "{fresh}"); + } + } + + /// Through the real handlers and a real `nats-server`: an entry lands in + /// the one value a hive reads, the two scopes of one name are two entries, + /// a repeat changes nothing, and a removal takes only the scope it names. + /// + /// The published bytes are pinned literally because hive-c0re's + /// `workers::squid_allow` tests decode this same string; a change to either + /// side's reading of it fails one of the two. + #[tokio::test] + async fn the_route_writes_the_list_a_hive_reads_and_each_scope_is_its_own_entry() { + let nats = TestNats::start(); + let client = nats.client().await; + let state = declared(&client).await; + + add(&state, "github.com", None).await.expect("swarm-wide"); + add(&state, "crates.io", Some("atlas")) + .await + .expect("for atlas"); + add(&state, "github.com", Some("atlas")) + .await + .expect("the same name for atlas alone"); + let before = published(&client).await; + add(&state, "GITHUB.com", None).await.expect("a repeat"); + assert_eq!(published(&client).await, before, "a repeat writes nothing"); + + assert_eq!( + published(&client).await.as_deref(), + Some( + r#"{"entries":[{"domain":"crates.io","agent":"atlas"},{"domain":"github.com"},{"domain":"github.com","agent":"atlas"}]}"# + ) + ); + assert_eq!( + listed(&state).await, + [ + ("crates.io".to_owned(), Some("atlas".to_owned())), + ("github.com".to_owned(), None), + ("github.com".to_owned(), Some("atlas".to_owned())), + ] + ); + + remove(&state, "github.com", Some("atlas")) + .await + .expect("removes atlas's"); + assert_eq!( + listed(&state).await, + [ + ("crates.io".to_owned(), Some("atlas".to_owned())), + ("github.com".to_owned(), None), + ] + ); + // Removing what is not there is not an error and writes nothing. + let before = published(&client).await; + remove(&state, "github.com", Some("atlas")) + .await + .expect("already gone"); + assert_eq!(published(&client).await, before); + + remove(&state, "github.com", None) + .await + .expect("swarm-wide"); + remove(&state, "crates.io", Some("atlas")) + .await + .expect("atlas's"); + assert_eq!( + published(&client).await.as_deref(), + Some(r#"{"entries":[]}"#), + "the last removal leaves an empty list, which still replaces a hive's own" + ); + } + + #[test] + fn a_full_list_takes_no_new_entry_but_a_present_one_is_not_a_new_one() { + let full = AllowList { + entries: (0..MAX_ENTRIES) + .map(|i| Entry { + domain: format!("host{i}.example.com"), + agent: None, + }) + .collect(), + }; + let new = Entry { + domain: "github.com".to_owned(), + agent: None, + }; + let why = added(full.clone(), &new).err().expect("a full list"); + assert!(why.contains("1000"), "{why}"); + // Controls: an entry already there is not refused, and one fewer fits. + let present = full.entries[0].clone(); + assert!(matches!( + added(full.clone(), &present), + Ok(Added::Already(_)) + )); + let mut room = full; + room.entries.pop(); + let Ok(Added::Now(list)) = added(room, &new) else { + panic!("a list with room takes the entry"); + }; + assert_eq!(list.entries.len(), MAX_ENTRIES); + } + + /// A read before anything was added answers an empty list and creates no + /// bucket, so a hive keeps its own list. + #[tokio::test] + async fn a_read_before_any_entry_answers_empty_and_creates_no_bucket() { + let nats = TestNats::start(); + let client = nats.client().await; + let state = state(&client); + + assert_eq!(listed(&state).await, []); + assert!( + egress_allow::open(&client) + .await + .expect("the queue answers") + .is_none() + ); + } + + /// An agent no hive declares gets no entry, so a typo cannot allow a name + /// for an agent that does not exist yet; an unscoped add is not asked. + #[tokio::test] + async fn an_agent_no_hive_declares_is_refused_and_nothing_is_written() { + let nats = TestNats::start(); + let client = nats.client().await; + let state = declared(&client).await; + + let problem = add(&state, "github.com", Some("iris")) + .await + .expect_err("no hive declares iris"); + assert_eq!(problem.status, Some(StatusCode::BAD_REQUEST), "{problem:?}"); + assert_eq!(published(&client).await, None); + add(&state, "github.com", Some("atlas")) + .await + .expect("control: a declared agent is accepted"); + } + + async fn post_raw(app: &axum::Router, body: &'static str) -> StatusCode { + use tower::ServiceExt as _; + app.clone() + .oneshot( + axum::http::Request::post("/api/egress/allowed-domains") + .header(axum::http::header::CONTENT_TYPE, "application/json") + .body(axum::body::Body::from(body)) + .expect("request"), + ) + .await + .expect("router answers") + .status() + } + + /// Through the real router: an IP literal or a wildcard is a 400 and + /// writes nothing, an unknown field a 422, and the well-formed body after + /// them is the control that the same route reaches the bucket. + #[tokio::test] + async fn the_router_refuses_an_ip_literal_and_a_wildcard_before_the_queue() { + let nats = TestNats::start(); + let client = nats.client().await; + let app = crate::startup::build_app(declared(&client).await); + + for body in [ + r#"{"domain":"1.2.3.4"}"#, + r#"{"domain":"10.0.0.1","agent":"atlas"}"#, + r#"{"domain":"*.example.com"}"#, + r#"{"domain":"[::1]"}"#, + r#"{"domain":""}"#, + ] { + assert_eq!( + post_raw(&app, body).await, + StatusCode::BAD_REQUEST, + "{body}" + ); + } + assert_eq!( + post_raw(&app, r#"{"domain":"github.com","hive":"pr1ma"}"#).await, + StatusCode::UNPROCESSABLE_ENTITY + ); + assert_eq!(published(&client).await, None); + + assert_eq!( + post_raw(&app, r#"{"domain":"github.com"}"#).await, + StatusCode::OK + ); + assert_eq!( + published(&client).await.as_deref(), + Some(r#"{"entries":[{"domain":"github.com"}]}"#) + ); + } +} diff --git a/swarm-controller/src/main.rs b/swarm-controller/src/main.rs index fc020d39..28d46b51 100644 --- a/swarm-controller/src/main.rs +++ b/swarm-controller/src/main.rs @@ -44,6 +44,7 @@ mod builder; mod cache_miss; mod claude_login; mod config_pr; +mod egress_allow; mod forge; mod forge_account; mod forge_events; diff --git a/swarm-controller/src/startup.rs b/swarm-controller/src/startup.rs index a24dab0b..307a0ba8 100644 --- a/swarm-controller/src/startup.rs +++ b/swarm-controller/src/startup.rs @@ -30,10 +30,10 @@ use crate::routes::{self, AppState, agents, config_prs, forge_admin, hives, jobq use crate::{ agent_control, agent_icon, agent_input, agent_limits, agent_login, agent_permissions, agent_renewal, agent_state_stream, agent_status, agents_md, alerts, archetypes, auth, - backend_env, builder, claude_login, config_pr, forge, forge_account, github_account, inference, - issue_report, linked_accounts, matrix_account, message_stream, notify_flows, notify_router_api, - pulls, queue_identity, review_classifier, schedules, status, subagent_term, term_stream, - wanted, webhook, + backend_env, builder, claude_login, config_pr, egress_allow, forge, forge_account, + github_account, inference, issue_report, linked_accounts, matrix_account, message_stream, + notify_flows, notify_router_api, pulls, queue_identity, review_classifier, schedules, status, + subagent_term, term_stream, wanted, webhook, }; /// Where the daemon binds, overridable via `SWARM_CONTROLLER_SOCKET`. @@ -77,6 +77,7 @@ pub(crate) fn socket_path() -> PathBuf { (name = "repos", description = "forge repo browsing (swarm-ui's issue-report page)"), (name = "forge", description = "forge site admins (`swarmctl forge make-admin`)"), (name = "schedules", description = "scheduled prompts (`swarmctl schedule`)"), + (name = "egress", description = "the destinations agents may reach through their hive's egress proxy"), (name = "notify", description = "the notification router's flows: built-ins switched off, and custom flows"), (name = "review-classifier", description = "trying review classifications against a PR (swarm-ui's try-out page), and the classifier's swarm-wide settings"), (name = "pulls", description = "open pull requests, for the try-out page's dropdowns"), @@ -620,6 +621,11 @@ pub(crate) fn build_app(state: AppState) -> axum::Router { agent_permissions::delete_agent_permissions )) .routes(routes!(agent_permissions::get_permission_catalogue)) + .routes(routes!( + egress_allow::get_allowed_domains, + egress_allow::add_allowed_domain, + egress_allow::remove_allowed_domain + )) .routes(routes!(archetypes::get_archetypes)) .routes(routes!(config_prs::get_config_prs)) .routes(routes!(agents::create_agent)) diff --git a/swarm-nats-auth/src/policy.rs b/swarm-nats-auth/src/policy.rs index 33d85dc8..9acdc172 100644 --- a/swarm-nats-auth/src/policy.rs +++ b/swarm-nats-auth/src/policy.rs @@ -610,6 +610,9 @@ impl Policy { "$JS.API.DIRECT.GET.{}.>", swarm_wire::agent_permissions::STREAM )); + // The egress allow-list (`swarm_wire::egress_allow`): one key, read with + // one direct get, so exactly that subject and no other key of the bucket. + subjects.push(swarm_wire::egress_allow::direct_get_subject()); // The archetype an agent was created from // (`swarm_queue_client::agent_archetype`), read the same way and every // key for the same reason. @@ -955,6 +958,32 @@ mod tests { } } + /// A hive reads the swarm's egress allow-list and can change none of it, + /// and no other key of its bucket. + #[test] + fn a_hive_may_read_the_egress_allow_list_and_write_none() { + use swarm_wire::egress_allow::{STREAM, direct_get_subject, key_subject}; + + let p = policy() + .permissions("hive-alpha") + .expect("a hive is admitted"); + let may = |subject: &str| p.publish.iter().any(|g| covers(g, subject)); + + assert!(may(&direct_get_subject()), "cannot read the list"); + for forbidden in [ + key_subject(), + format!("$JS.API.STREAM.CREATE.{STREAM}"), + format!("$JS.API.STREAM.UPDATE.{STREAM}"), + format!("$JS.API.STREAM.DELETE.{STREAM}"), + format!("$JS.API.STREAM.PURGE.{STREAM}"), + format!("$JS.API.CONSUMER.CREATE.{STREAM}"), + // Another key of the same bucket. + format!("$JS.API.DIRECT.GET.{STREAM}.$KV.egress-allow.other"), + ] { + assert!(!may(&forbidden), "hive alpha may publish {forbidden}"); + } + } + /// A hive reads any agent's archetype and can change none of them, and the /// grant reaches no other bucket. #[test] diff --git a/swarm-queue-client/Cargo.toml b/swarm-queue-client/Cargo.toml index 6538afba..bd77e898 100644 --- a/swarm-queue-client/Cargo.toml +++ b/swarm-queue-client/Cargo.toml @@ -51,6 +51,9 @@ operator-profiles = ["async-nats/jetstream"] # `agent_limits::read`, for hive-c0re, which reads the key of each agent it # runs the same way. Explicit for the same reason as `notices`. agent-limits = ["async-nats/jetstream"] +# `egress_allow::read`, for hive-c0re, which reads the swarm's egress allow-list +# with one direct get. Explicit for the same reason as `notices`. +egress-allow = ["async-nats/jetstream"] # `agent_permissions::read` and `held`, for hive-c0re, the agent # harness, the subagent daemon and swarm-controller. Pulls `hive-types` for the # `Permission` names, so the crates that only need the bucket name do not link diff --git a/swarm-queue-client/src/egress_allow.rs b/swarm-queue-client/src/egress_allow.rs new file mode 100644 index 00000000..90b280f0 --- /dev/null +++ b/swarm-queue-client/src/egress_allow.rs @@ -0,0 +1,98 @@ +//! The connection side of the egress allow-list KV bucket: opening it for +//! writing or reading, and the hive's direct read. The bucket's names, subjects +//! and stored type are `swarm_wire::egress_allow`. + +/// Open the allow-list bucket for writing, creating it if nothing has yet. +/// +/// **Controller-side only.** `history: 1`: a hive wants the current value. +#[cfg(feature = "kv")] +pub async fn open_or_create( + client: &async_nats::Client, +) -> Result { + let js = async_nats::jetstream::new(client.clone()); + crate::kv::open_or_create( + &js, + swarm_wire::egress_allow::BUCKET, + "egress-allow", + "The destinations the swarm lets agents reach through their hive's proxy".to_owned(), + ) + .await +} + +/// Open the allow-list bucket for reading, or `None` when nothing has created +/// it yet, which reads the same as no list. +/// +/// **Controller-side**, for reads that must not create the bucket. +/// +/// # Errors +/// +/// [`crate::Error::OpenBucket`] for every failure other than the bucket not +/// existing. +#[cfg(feature = "kv")] +pub async fn open( + client: &async_nats::Client, +) -> Result, crate::Error> { + let js = async_nats::jetstream::new(client.clone()); + match js.get_key_value(swarm_wire::egress_allow::BUCKET).await { + Ok(store) => Ok(Some(store)), + Err(e) if crate::wanted::is_bucket_not_found(&e) => Ok(None), + Err(source) => Err(crate::Error::OpenBucket { + bucket: swarm_wire::egress_allow::BUCKET.to_owned(), + source, + }), + } +} + +/// What reading the list found. +#[cfg(feature = "egress-allow")] +#[derive(Debug)] +pub enum Read { + /// The controller has written a list, which may be empty. + Current(swarm_wire::egress_allow::AllowList), + /// There is no list to read: no bucket answers, or the key was never + /// written or has been deleted. A controller that predates the bucket reads + /// this way, and so does a queue that is not serving the stream right now, + /// so it says nothing about what the swarm wants. + NotSet, +} + +/// Read the current list with one direct get. +/// +/// # Errors +/// +/// A failed request or an undecodable value. A request the queue refused for +/// permissions reaches the client as a timeout, not as a refusal. +#[cfg(feature = "egress-allow")] +pub async fn read( + client: &async_nats::Client, +) -> Result> { + use async_nats::jetstream::stream::DirectGetErrorKind; + use swarm_wire::egress_allow::{STREAM, key_subject}; + + let js = async_nats::jetstream::new(client.clone()); + let stream = js.get_stream_no_info(STREAM).await?; + let message = match stream.direct_get_last_for_subject(key_subject()).await { + Ok(message) => message, + Err(e) if matches!(e.kind(), DirectGetErrorKind::NotFound) => return Ok(Read::NotSet), + Err(e) if is_no_responders(&e) => return Ok(Read::NotSet), + Err(e) => return Err(e.into()), + }; + // A KV delete or purge is stored as an empty marker carrying this header. + let removed = message + .headers + .get("KV-Operation") + .is_some_and(|op| matches!(op.as_str(), "DEL" | "PURGE")); + if removed { + return Ok(Read::NotSet); + } + Ok(Read::Current(serde_json::from_slice(&message.payload)?)) +} + +#[cfg(feature = "egress-allow")] +fn is_no_responders(e: &async_nats::jetstream::stream::DirectGetError) -> bool { + matches!( + e.kind(), + async_nats::jetstream::stream::DirectGetErrorKind::ErrorResponse(status, _) + if status == async_nats::StatusCode::NO_RESPONDERS + ) +} diff --git a/swarm-queue-client/src/lib.rs b/swarm-queue-client/src/lib.rs index f7fe0ec6..718634d6 100644 --- a/swarm-queue-client/src/lib.rs +++ b/swarm-queue-client/src/lib.rs @@ -232,6 +232,12 @@ pub mod agent_inference; /// and reading needs `agent-limits`. pub mod agent_limits; +/// Opening and reading the bucket swarm-controller keeps the egress allow-list +/// in — one key, read by every hive. The names, the stored type and the +/// validation are `swarm_wire::egress_allow`; writing needs `kv` and reading +/// needs `egress-allow`. +pub mod egress_allow; + /// Opening and reading the bucket swarm-controller sets each agent's /// permissions in — one key per agent, read by the hive that runs it. The names, /// the stored type and the import messages are `swarm_wire::agent_permissions`; diff --git a/swarm-wire/src/egress_allow.rs b/swarm-wire/src/egress_allow.rs new file mode 100644 index 00000000..195c83e0 --- /dev/null +++ b/swarm-wire/src/egress_allow.rs @@ -0,0 +1,241 @@ +//! The egress allow-list KV bucket: one key holding the destination names the +//! swarm lets agents reach through their hive's squid. +//! +//! **swarm-controller is the only writer**, and creates the bucket on its +//! first write. Each [`Entry`] is a domain with a scope: every agent in the +//! swarm, or one agent. Until the controller has written the key, a hive +//! keeps the list its own nix configuration names. +//! +//! **Every hive reads the one key** ([`direct_get_subject`]), with a grant that +//! covers the bucket read-only, and renders the entries for the agents it runs. +//! The names go into squid's configuration, so [`normalize_domain`] is the one +//! gate both ends use: the controller at write, the hive again at render. + +/// The KV bucket the list is published into. +pub const BUCKET: &str = "egress-allow"; + +/// The stream backing [`BUCKET`]. `create_key_value` prefixes `KV_`. +pub const STREAM: &str = "KV_egress-allow"; + +/// The key the whole list is stored under. +pub const KEY: &str = "list"; + +/// The subject the list is stored under. +#[must_use] +pub fn key_subject() -> String { + format!("$KV.{BUCKET}.{KEY}") +} + +/// The request subject that reads the current list. +#[must_use] +pub fn direct_get_subject() -> String { + format!("$JS.API.DIRECT.GET.{STREAM}.{}", key_subject()) +} + +/// The most entries a list holds, which keeps the value far below the queue's +/// message size and the rendered squid configuration small. +pub const MAX_ENTRIES: usize = 1000; + +/// One allowed destination and who may reach it. +#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, serde::Serialize, serde::Deserialize)] +pub struct Entry { + /// A name as [`normalize_domain`] returns it. + pub domain: String, + /// The agent this entry is for; absent means every agent in the swarm. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub agent: Option, +} + +/// The value under [`BUCKET`]/[`KEY`]. No `deny_unknown_fields`: a field a +/// newer controller adds must not cost an older hive the entries it reads. +#[derive(Debug, Clone, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)] +pub struct AllowList { + #[serde(default)] + pub entries: Vec, +} + +/// `raw` as the name stored and rendered, or why it is refused. +/// +/// A DNS name of at least two labels, lower-cased, with at most one leading +/// dot. The dot is squid's `dstdomain` form for the name and all its +/// subdomains. Refused: IP literals (a final label that does not start with a +/// letter), single labels, wildcards, ports, paths, whitespace inside, a +/// trailing dot and anything outside `[a-z0-9.-]`. +/// +/// # Errors +/// A caller-ready message naming the rule the input breaks. +pub fn normalize_domain(raw: &str) -> Result { + let domain = raw.trim().to_ascii_lowercase(); + let name = domain.strip_prefix('.').unwrap_or(&domain); + if name.len() > 253 { + return Err(format!("{raw:?} is longer than 253 characters")); + } + let labels: Vec<&str> = name.split('.').collect(); + if labels.len() < 2 { + return Err(format!( + "{raw:?} is not a domain name: it needs at least two labels, such as example.com" + )); + } + for label in &labels { + let ok = (1..=63).contains(&label.len()) + && label + .bytes() + .all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'-') + && !label.starts_with('-') + && !label.ends_with('-'); + if !ok { + return Err(format!( + "{raw:?} is not a domain name: labels are 1 to 63 letters, digits or hyphens, \ + not starting or ending with a hyphen, joined by dots; wildcards, ports and \ + paths are refused, and the only dot form is one leading dot for the name and \ + its subdomains" + )); + } + } + if !labels + .last() + .is_some_and(|tld| tld.starts_with(|c: char| c.is_ascii_lowercase())) + { + return Err(format!( + "{raw:?} is not a domain name: the last label must start with a letter, which \ + also refuses IP addresses" + )); + } + Ok(domain) +} + +/// Whether squid's `dstdomain` list `names` already matches everything +/// `domain` does. An exact name matches itself; a name with a leading dot also +/// matches its subdomains. Both sides are in [`normalize_domain`]'s form. +#[must_use] +pub fn covered_by>(domain: &str, names: &[S]) -> bool { + names.iter().any(|n| { + let n = n.as_ref(); + match (domain.strip_prefix('.'), n.strip_prefix('.')) { + // Every subdomain of the entry is a subdomain of the list's name. + (Some(d), Some(base)) => d == base || d.ends_with(&format!(".{base}")), + // An exact name never covers a name that has subdomains. + (Some(_), None) => false, + (None, Some(base)) => domain == base || domain.ends_with(&format!(".{base}")), + (None, None) => domain == n, + } + }) +} + +#[cfg(test)] +mod tests { + use super::*; + + /// The names the hive's grant in `swarm-nats-auth` is built from. + #[test] + fn names_match_the_queue_grant() { + assert_eq!(STREAM, format!("KV_{BUCKET}")); + assert_eq!(key_subject(), "$KV.egress-allow.list"); + assert_eq!( + direct_get_subject(), + "$JS.API.DIRECT.GET.KV_egress-allow.$KV.egress-allow.list" + ); + } + + #[test] + fn a_dns_name_is_kept_lower_cased_with_one_optional_leading_dot() { + for (raw, stored) in [ + ("github.com", "github.com"), + (" GitHub.COM ", "github.com"), + (".githubusercontent.com", ".githubusercontent.com"), + ("a-b.c1.example.co.uk", "a-b.c1.example.co.uk"), + ("xn--bcher-kva.example", "xn--bcher-kva.example"), + ] { + assert_eq!(normalize_domain(raw).as_deref(), Ok(stored), "{raw:?}"); + } + } + + #[test] + fn an_ip_literal_a_wildcard_or_anything_but_a_name_is_refused() { + for bad in [ + "", + ".", + "localhost", + ".com", + "1.2.3.4", + ".1.2.3.4", + "127.0.0.1", + "0x7f.1", + "0x7f.0x1", + "[::1]", + "::1", + "fd00::1", + "*.example.com", + "*", + "..example.com", + "example..com", + "example.com.", + "example.com:443", + "example.com/path", + "https://example.com", + "exa mple.com", + "example.com\nhttp_access allow all", + "-a.example.com", + "a-.example.com", + "a_b.example.com", + "bücher.example", + "example.123", + ] { + assert!(normalize_domain(bad).is_err(), "{bad:?} was accepted"); + } + let long = format!("{}.example.com", "a".repeat(64)); + assert!(normalize_domain(&long).is_err(), "a 64-character label"); + let too_long = format!("{}.com", vec!["a".repeat(60); 5].join(".")); + assert!(normalize_domain(&too_long).is_err(), "over 253 characters"); + // Control: the boundaries just inside the limits are names. + assert!(normalize_domain(&format!("{}.example.com", "a".repeat(63))).is_ok()); + } + + #[test] + fn covered_by_follows_squids_dstdomain_matching() { + let base = ["api.anthropic.com", ".claude.ai"]; + assert!(covered_by("api.anthropic.com", &base)); + assert!( + covered_by("claude.ai", &base), + "the dot form covers the apex" + ); + assert!(covered_by("a.claude.ai", &base)); + assert!(covered_by(".a.claude.ai", &base), "a narrower dot form"); + assert!(covered_by(".claude.ai", &base)); + // An exact base name does not cover its subdomains or a dot form. + assert!(!covered_by("x.api.anthropic.com", &base)); + assert!(!covered_by(".api.anthropic.com", &base)); + // A broader dot form is new access. + assert!(!covered_by(".anthropic.com", &base)); + assert!( + !covered_by("notclaude.ai", &base), + "a suffix is not a subdomain" + ); + assert!(!covered_by("github.com", &base)); + assert!(!covered_by("github.com", &[] as &[&str])); + } + + #[test] + fn the_stored_value_leaves_out_what_is_unset_and_tolerates_new_fields() { + let list = AllowList { + entries: vec![ + Entry { + domain: "github.com".to_owned(), + agent: None, + }, + Entry { + domain: "crates.io".to_owned(), + agent: Some("atlas".to_owned()), + }, + ], + }; + let wire = + r#"{"entries":[{"domain":"github.com"},{"domain":"crates.io","agent":"atlas"}]}"#; + assert_eq!(serde_json::to_string(&list).unwrap(), wire); + assert_eq!(serde_json::from_str::(wire).unwrap(), list); + assert_eq!( + serde_json::from_str::(r#"{"entries":[],"later":1}"#).unwrap(), + AllowList::default() + ); + } +} diff --git a/swarm-wire/src/lib.rs b/swarm-wire/src/lib.rs index 1ce77d04..995fdf14 100644 --- a/swarm-wire/src/lib.rs +++ b/swarm-wire/src/lib.rs @@ -17,6 +17,7 @@ pub mod agent_status; pub mod agent_term; pub mod builder; pub mod claude_login; +pub mod egress_allow; pub mod forge_events; pub mod notices; pub mod notify_flows;