vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

make squid's allowed domains editable at runtime in the swarm ui

Browse source
This commit is contained in:
atlas 2026-10-28 20:39:22 +01:00 • committed by mara
commit c9abc2a42b
30 changed files with 2307 additions and 35 deletions
Download patch file Download diff file Expand all files Collapse all files

37
docs/networking/network.md
View file

@ -306,8 +306,43 @@ host. It never decrypts TLS, so it holds no signing key. It caches nothing.
the canary widens the restricted list by that one name. The swarm's own
service names aren't on it: they resolve to the bridge or a private address,
which the rules below refuse, and agents reach them directly. Each entry
matches that exact name. A user on the unrestricted list reaches any
matches that exact name, or with a leading dot the name and its subdomains. A user on the unrestricted list reaches any
other name too, on ports 80 and 443. Everyone else gets a 403 for it.
`allowedDomains` is the hive's own list. Squid uses it until the swarm has a
list of its own (next bullet).
- **Swarm allow-list.** The operator adds domains in the swarm UI
([allowed domains](../swarm/ui.md#allowed-domains)); the swarm controller
keeps them in one value of the `egress-allow` bucket, each for every agent
or for one agent. Every 30 seconds `hive-c0re` reads it and writes two files
beside the users file, `services.hyperhive.squid.allowedFile`
(`/var/lib/hive-squid/allowed`) and `services.hyperhive.squid.agentAllowFile`
(`/var/lib/hive-squid/agent-allow.conf`), `hive-core:squid`, `0640`, and
logs `squid allow list: 2 swarm, 3 agent entries` when they change. The
swarm-wide names join the allow ACL, so a restricted user reaches them. A
per-agent entry is a rule for that agent's squid user, `http_access allow
<user> <names>`, in the generated `agent-allow.conf` that the configuration
includes at the point where the base list gets its allow: after every
destination deny and
`deny !authed`, so the rules under _Denied before any allow_ still bind it,
and before the unrestricted rule. Only this hive's agents get a rule.
`squid-unrestricted-reload.path` reloads squid when either file changes, so
there is no deploy or restart.
A name is a DNS name of at least two labels. One leading dot (`.example.com`)
also allows every subdomain, and is the only wildcard; the controller refuses
IP addresses, other wildcards, ports, paths and the base list's names when
you add the entry, and `hive-c0re` checks each name again and skips one that
fails, or that the base list or the canary covers, or that another entry of
the same list covers. A name that resolves to a private address stays a 403.
The swarm's list replaces the hive's `allowedDomains` as soon as it exists,
even when empty: add the names the hive allowed to the swarm's list
before relying on it. Until the controller has a list (it predates the
feature, or nobody added an entry) `allowedFile` holds `allowedDomains` and
`agent-allow.conf` is empty. A queue that stops answering for the bucket
leaves a file the swarm wrote untouched, and a failed read leaves both
files untouched. The first line of a file written from the swarm's list is a
`# source: swarm` comment. The queue grant a hive needs to read the list
comes with the controller's `swarm-nats-auth`; a hive that can't read it
times out, keeps its files and warns after about 90 seconds.
- **Unlisted names.** For a user who isn't on the unrestricted list, refused
after the destination checks, which look the name up on the host.
- **Denied before any allow.** Destinations that resolve to loopback,

34
docs/swarm/ui.md
View file

@ -112,7 +112,7 @@ with no icon keeps the hyperhive mark.
The tab also has an agent badge, with the agent's icon and name, ahead of the
connection badge. It opens the agent's details in a popover under the badge:
the same status, state, wanted state, inference settings, resource caps, permissions and linked accounts as the
the same status, state, wanted state, inference settings, resource caps, permissions, allowed domains and linked accounts as the
panel on `/agents`, without the terminal preview. The popover doesn't dim the
page. Selecting outside it, or Escape, closes it. From the keyboard, opening a
popover moves focus into it, and Escape puts focus back on the badge. The filter
@ -622,6 +622,38 @@ restarts the agent after its current turn. The panel reads
the reason and a **retry** button. `swarmctl agent permissions set` sets the
same grant from a shell.
### Allowed domains
The agent's detail panel lists the destinations it may reach through its
hive's egress proxy under **allowed domains**, beyond the base list every
agent has: the entries for every agent in the swarm (badge **swarm-wide**) and
the ones for this agent alone (badge with the agent's name). An entry matters
to a restricted agent. One with the unrestricted internet flag reaches any
name the proxy's private-range rules leave. **remove** drops one entry;
**add** opens a dialog with a domain box and a tick for every agent in the
swarm. The tick is off by default, so an entry is for this agent alone.
A domain is a DNS name of at least two labels, such as `github.com`. One
leading dot, as in `.githubusercontent.com`, also allows every subdomain; the
dot form is the only wildcard, because exact names can't list a service that
spreads over many hostnames, and an entry is the operator's choice. The
controller refuses IP addresses, other wildcards, ports and paths, and a name
the proxy already allows every agent (the base list). The dialog shows the
refusal's reason under the box and leaves what you typed. Adding does a
`POST /api/egress/allowed-domains` and removing a `DELETE` of the same path,
each with `{"domain": "github.com", "agent": "atlas"}` (leave out `agent` for
a swarm-wide entry); both answer the whole list, and `GET` reads it. Adding
an entry that exists changes nothing.
A saved change isn't instant. Each hive reads the list about every 30
seconds and reloads its proxy, so the name is reachable within a minute. The
proxy's private-range rules still apply: an entry whose name resolves to a
loopback, private or bridge address stays blocked. The first entry makes the
swarm's list replace the names a hive's own `allowedDomains` lists, so add
those names here too; a hive keeps its own list until then. See [the egress
proxy](../networking/network.md#egress-proxy-squid) for what each hive does
when the controller is old or unreachable.
### Creating an agent
The **+ agent** button opens a dialog of five steps, which you go through in

5
frontend/packages/swarm-ui/src/agent/detail/AgentDetail.tsx
View file

@ -14,6 +14,7 @@ import { AgentInference } from "./AgentInference.js";
import { AgentInstructions } from "./AgentInstructions.js";
import { AgentLimits } from "./AgentLimits.js";
import { AgentPermissions } from "./AgentPermissions.js";
import { AllowedDomains } from "./AllowedDomains.js";
import { AgentStateRow } from "./AgentStateRow.js";
import { AgentTermPreview } from "../terminal/AgentTermPreview.js";
import { FRESHNESS, type AgentRow } from "../AgentTypes.js";
@ -122,6 +123,10 @@ export function AgentDetail({
<dd>
<AgentPermissions key={row.name} agent={row.name} />
</dd>
<dt>allowed domains</dt>
<dd>
<AllowedDomains key={row.name} agent={row.name} />
</dd>
<dt>accounts</dt>
<dd>
{row.hive ? (

243
frontend/packages/swarm-ui/src/agent/detail/AllowedDomains.tsx Normal file
View file

@ -0,0 +1,243 @@
// <AllowedDomains> — the destinations the swarm lets one agent reach through
// its hive's egress proxy, beyond the base list every agent has: the entries
// that hold for every agent and the ones for this agent alone, with an editor
// to add and remove them.
//
// Reads `GET /api/egress/allowed-domains`, which answers every entry, and
// shows the ones that apply to this agent. Adding POSTs one entry and removing
// DELETEs one, each with the entry as the body, each answering the whole list.
// The route's refusal (an IP address, a wildcard, a name the proxy already
// allows everyone) shows as it arrives, next to the box. The hive reads a
// change within about 30 seconds.
//
// The first entry makes the swarm's list replace the names the hive's own
// configuration allows, so a name the hive allowed and the swarm's list lacks
// stops being allowed once hives read it. An agent that holds the unrestricted
// internet flag reaches any name the proxy's private-range rules leave, so an
// entry matters only to a restricted one.
//
// Its state belongs to one agent: callers key it by agent name.
import { useEffect, useState } from "preact/hooks";
import { ApiErrorPanel } from "@hive/shared/api-error-panel.js";
import { readApiError, type ProblemDetails } from "@hive/shared/api-error.js";
import { Badge } from "@hive/shared/badge.js";
import { Button } from "../../ui/button/Button.js";
import { CheckboxField } from "../../ui/checkbox-field/CheckboxField.js";
import { Dialog } from "../../ui/dialog/Dialog.js";
import { Panel } from "../../ui/panel/Panel.js";
import { TextField } from "../../ui/text-field/TextField.js";
import {
ALLOWED_DOMAINS_URL,
allowedSummary,
entriesFor,
entryBody,
type AllowedDomain,
} from "./allowed-domains.js";
import "./AgentLimits.css";
interface Listing {
entries: AllowedDomain[];
}
export function AllowedDomains({ agent }: { agent: string }) {
const [entries, setEntries] = useState<AllowedDomain[] | null>(null);
const [error, setError] = useState<ProblemDetails | null>(null);
// Bumped after an add or a removal, so the list is fetched again.
const [reload, setReload] = useState(0);
const [adding, setAdding] = useState(false);
const [busy, setBusy] = useState(false);
useEffect(() => {
let cancelled = false;
(async () => {
const r = await fetch(ALLOWED_DOMAINS_URL);
if (!r.ok) {
if (!cancelled) setError(await readApiError(r));
return;
}
const body = (await r.json()) as Listing;
if (cancelled) return;
setEntries(body.entries);
setError(null);
})().catch((e: unknown) => {
if (!cancelled) setError({ detail: String(e) });
});
return () => {
cancelled = true;
};
}, [reload]);
async function remove(entry: AllowedDomain) {
if (busy) return;
setBusy(true);
try {
const r = await fetch(ALLOWED_DOMAINS_URL, {
method: "DELETE",
headers: { "content-type": "application/json" },
body: JSON.stringify(entry),
});
if (!r.ok) {
setError(await readApiError(r));
return;
}
setReload((n) => n + 1);
} catch (e: unknown) {
setError({ detail: String(e) });
} finally {
setBusy(false);
}
}
// A list already loaded stays under a failed refetch, so a change that
// worked is not hidden by a read that did not.
const errorPanel = error ? (
<>
<ApiErrorPanel
context="the allowed domains were not read or changed"
problem={error}
/>
<Button onClick={() => setReload((n) => n + 1)}>retry</Button>
</>
) : null;
if (entries === null) {
return (
errorPanel ?? (
<span class="ui-agent-limits-muted" role="status">
loading the allowed domains…
</span>
)
);
}
const shown = entriesFor(entries, agent);
return (
<div class="ui-agent-limits">
{errorPanel}
<Badge label="allowed domains" value={allowedSummary(shown)} />
{shown.length > 0 ? (
<ul class="ui-agent-limits-list">
{shown.map((e) => (
<li key={`${e.agent ?? "*"}/${e.domain}`}>
<Badge
label={e.agent === undefined ? "swarm-wide" : agent}
value={e.domain}
/>{" "}
<Badge
variant="quiet"
value="remove"
title={`stop allowing ${e.domain} for ${
e.agent === undefined ? "every agent" : agent
}`}
onClick={() => void remove(e)}
/>
</li>
))}
</ul>
) : null}
<Badge
variant="quiet"
value="add"
title={`allow a domain for ${agent} or for every agent`}
onClick={() => setAdding(true)}
/>
<Dialog
open={adding}
onClose={() => setAdding(false)}
label="add an allowed domain"
plain
>
{/* Remounted per open, so a cancelled add leaves nothing behind. */}
{adding ? (
<AddForm
agent={agent}
onClose={() => setAdding(false)}
onAdded={() => {
setAdding(false);
setReload((n) => n + 1);
}}
/>
) : null}
</Dialog>
</div>
);
}
function AddForm({
agent,
onClose,
onAdded,
}: {
agent: string;
onClose: () => void;
onAdded: () => void;
}) {
const [domain, setDomain] = useState("");
const [forSwarm, setForSwarm] = useState(false);
const [busy, setBusy] = useState(false);
const [problem, setProblem] = useState<ProblemDetails | null>(null);
async function save(e: Event) {
e.preventDefault();
setBusy(true);
setProblem(null);
try {
const r = await fetch(ALLOWED_DOMAINS_URL, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify(entryBody(domain, forSwarm, agent)),
});
if (!r.ok) {
// The box stays as typed, so the operator can fix it and add again.
setProblem(await readApiError(r));
return;
}
onAdded();
} catch (err: unknown) {
setProblem({ detail: String(err) });
} finally {
setBusy(false);
}
}
return (
<Panel title={`allowed domain — ${agent}`} icon="🌐" onClose={onClose}>
<p>
Lets {agent} reach one more name through its hive's proxy, or every
agent when the box below is ticked. A name needs at least two labels,
such as <code>github.com</code>; a leading dot, as in{" "}
<code>.githubusercontent.com</code>, also allows every subdomain. IP
addresses and wildcards are refused. A name that resolves to a private
address stays blocked. The first entry makes the swarm's list replace
the names each hive's own configuration allows. The hive reads a change
within about 30 seconds.
</p>
<form class="ui-agent-limits-form" onSubmit={(e) => void save(e)}>
<TextField
id="allowed-domain"
label="domain"
value={domain}
placeholder="e.g. github.com"
onInput={setDomain}
/>
<CheckboxField
id="allowed-domain-swarm"
label="for every agent in the swarm"
checked={forSwarm}
onChange={setForSwarm}
/>
{problem ? (
<ApiErrorPanel context="the domain was not added" problem={problem} />
) : null}
<div class="ui-agent-limits-actions">
<Button
variant="primary"
type="submit"
disabled={busy || domain.trim() === ""}
>
{busy ? "adding…" : "add"}
</Button>
</div>
</form>
</Panel>
);
}

52
frontend/packages/swarm-ui/src/agent/detail/allowed-domains.ts Normal file
View file

@ -0,0 +1,52 @@
// The detail panel's allowed-domains editor: which entries one agent's panel
// shows and the bodies it sends. Kept out of the component so `npm test` can
// run it.
/** One entry as `GET /api/egress/allowed-domains` carries it. */
export interface AllowedDomain {
domain: string;
/** Absent for an entry that holds for every agent in the swarm. */
agent?: string;
}
export const ALLOWED_DOMAINS_URL = "/api/egress/allowed-domains";
/**
* The entries that apply to `agent`: the swarm-wide ones first, then the ones
* for this agent alone. Entries for other agents are not shown on this panel.
*/
export function entriesFor(
entries: AllowedDomain[],
agent: string,
): AllowedDomain[] {
const swarmWide = entries.filter((e) => e.agent === undefined);
const own = entries.filter((e) => e.agent === agent);
return [...swarmWide, ...own];
}
/** What the panel says about a list of entries. */
export function allowedSummary(shown: AllowedDomain[]): string {
if (shown.length === 0) return "none added";
const swarmWide = shown.filter((e) => e.agent === undefined).length;
const own = shown.length - swarmWide;
return [
swarmWide > 0 ? `${swarmWide} swarm-wide` : null,
own > 0 ? `${own} for this agent` : null,
]
.filter((part) => part !== null)
.join(", ");
}
/**
* The body that adds or removes `domain`, as typed: the route lower-cases it
* and refuses what is not a name. `forSwarm` makes it hold for every agent;
* otherwise it is for `agent` alone.
*/
export function entryBody(
domain: string,
forSwarm: boolean,
agent: string,
): AllowedDomain {
const name = domain.trim();
return forSwarm ? { domain: name } : { domain: name, agent };
}

54
frontend/packages/swarm-ui/test/allowed-domains.test.mjs Normal file
View file

@ -0,0 +1,54 @@
// The detail panel's allowed-domains editor: which entries one agent sees and
// the bodies it sends. Run by `npm test` with Node's type stripping.
import assert from "node:assert/strict";
import { test } from "node:test";
import {
allowedSummary,
entriesFor,
entryBody,
} from "../src/agent/detail/allowed-domains.ts";
const all = [
{ domain: "crates.io", agent: "atlas" },
{ domain: "docs.rs", agent: "argus" },
{ domain: "github.com" },
{ domain: "pypi.org", agent: "atlas" },
];
test("an agent sees the swarm-wide entries first, then its own, and not another agent's", () => {
assert.deepEqual(entriesFor(all, "atlas"), [
{ domain: "github.com" },
{ domain: "crates.io", agent: "atlas" },
{ domain: "pypi.org", agent: "atlas" },
]);
// Control: the other agent's entry is there to be found.
assert.deepEqual(entriesFor(all, "argus"), [
{ domain: "github.com" },
{ domain: "docs.rs", agent: "argus" },
]);
assert.deepEqual(entriesFor(all, "iris"), [{ domain: "github.com" }]);
assert.deepEqual(entriesFor([], "atlas"), []);
});
test("the summary counts each scope", () => {
assert.equal(allowedSummary([]), "none added");
assert.equal(
allowedSummary(entriesFor(all, "atlas")),
"1 swarm-wide, 2 for this agent",
);
assert.equal(allowedSummary(entriesFor(all, "iris")), "1 swarm-wide");
assert.equal(
allowedSummary([{ domain: "a.io", agent: "atlas" }]),
"1 for this agent",
);
});
test("the body is swarm-wide or for the agent, with the name trimmed", () => {
assert.deepEqual(entryBody(" github.com ", true, "atlas"), {
domain: "github.com",
});
assert.deepEqual(entryBody("github.com", false, "atlas"), {
domain: "github.com",
agent: "atlas",
});
});

1
hive-c0re/Cargo.toml
View file

@ -62,6 +62,7 @@ swarm-queue-client = { workspace = true, features = [
"kv",
"notices",
"agent-limits",
"egress-allow",
"agent-permissions",
"agent-archetype",
"agent-closures",

6
hive-c0re/src/swarm_status.rs
View file

@ -116,6 +116,12 @@ pub fn spawn(
shutdown.clone(),
));
// The swarm's egress allow-list, on the same connection.
tokio::spawn(crate::workers::squid_allow::follow(
client.clone(),
shutdown.clone(),
));
// The permissions the swarm grants, and the import of the meta repo's.
tokio::spawn(crate::workers::swarm_permissions::follow(
client.clone(),

1
hive-c0re/src/workers/mod.rs
View file

@ -11,6 +11,7 @@ pub mod crash_watch;
pub mod knowledge;
pub mod mcp_sockets;
pub mod permissions_import;
pub mod squid_allow;
pub mod squid_users;
pub mod swarm_limits;
pub mod swarm_permissions;

596
hive-c0re/src/workers/squid_allow.rs Normal file
View file

@ -0,0 +1,596 @@
//! Squid's allow-list files: the destination names the swarm lets this hive's
//! agents reach through the egress proxy.
//!
//! swarm-controller keeps one list for the whole swarm
//! ([`swarm_wire::egress_allow`]); each entry is for every agent or for one.
//! Every [`INTERVAL`] this reads it and renders two files for squid:
//!
//! - [`ENV_ALLOWED_FILE`]: the swarm-wide names, one per line, which
//! `nix/host-modules/hive-squid.nix` reads as part of `allowed_dst`.
//! - [`ENV_AGENT_ALLOW_FILE`]: squid configuration `include`d where the base
//! list is allowed: for each of this hive's agents with entries, an `acl`
//! for its squid user, an `acl` for its names and the `http_access allow`
//! joining them. Entries of agents this hive does not run are left out.
//!
//! Squid reads both when it parses its configuration, so the path unit in
//! `nix/host-modules/hive-squid.nix` reloads it when either changes.
//!
//! The swarm's list replaces the names `services.hyperhive.squid.allowedDomains`
//! lists ([`ENV_SEED_FILE`]), which stay the list until the swarm has one. A
//! file rendered from the swarm's list starts with [`SWARM_MARKER`]. The queue
//! not answering for the bucket does not show that the swarm withdrew its list,
//! so a file with the marker is left alone then.
//!
//! Every entry is checked again with
//! [`swarm_wire::egress_allow::normalize_domain`], since the names end up in
//! squid's configuration, and skipped when it fails. An entry that
//! [`ENV_ALWAYS_ALLOWED`] or another entry of its list covers is dropped,
//! because squid ignores the narrower of two overlapping names in one ACL.
//!
//! A failed read leaves both files as they are.
use std::path::{Path, PathBuf};
use std::time::Duration;
use anyhow::{Context, Result};
use swarm_queue_client::egress_allow::{self, Read};
use swarm_wire::egress_allow::{AllowList, covered_by, normalize_domain};
use super::squid_users::write_if_changed;
use crate::stats::sweep_health::{self, SweepHealth};
/// How often the list is read.
pub const INTERVAL: Duration = Duration::from_secs(30);
/// At [`INTERVAL`] this is ~90 seconds of failing reads.
const FAILURES_BEFORE_WARNING: u32 = 3;
/// The swarm-wide names file. Set by `nix/host-modules/hive-c0re` from
/// `services.hyperhive.squid.allowedFile`.
pub const ENV_ALLOWED_FILE: &str = "HIVE_SQUID_ALLOWED_FILE";
/// The per-agent squid configuration file. Set from
/// `services.hyperhive.squid.agentAllowFile`.
pub const ENV_AGENT_ALLOW_FILE: &str = "HIVE_SQUID_AGENT_ALLOW_FILE";
/// The file of names `services.hyperhive.squid.allowedDomains` lists, one per
/// line, which the files hold until the swarm has a list.
pub const ENV_SEED_FILE: &str = "HIVE_SQUID_ALLOWED_SEED_FILE";
/// The names, separated by spaces, that squid allows every authenticated user
/// regardless of this list: the base list and the self-check canary.
pub const ENV_ALWAYS_ALLOWED: &str = "HIVE_SQUID_ALWAYS_ALLOWED";
/// The first line of a file rendered from the swarm's list.
const SWARM_MARKER: &str = "# source: swarm\n";
/// The first line of a file rendered from the seed.
const SEED_MARKER: &str = "# source: seed\n";
/// The two files a pass writes.
struct Files<'a> {
allowed: &'a Path,
agent_allow: &'a Path,
}
/// What a pass left in the files.
#[derive(Debug, PartialEq, Eq)]
enum Source {
/// The swarm's list.
Swarm,
/// The seed, since the swarm has no list.
Seed,
/// The swarm has no list now, but the files hold the one it had.
Kept,
}
fn env_path(name: &str) -> Option<PathBuf> {
std::env::var_os(name)
.filter(|v| !v.is_empty())
.map(PathBuf::from)
}
/// Read the list now and every [`INTERVAL`] until shutdown, when this host has
/// squid's two files.
pub async fn follow(client: async_nats::Client, mut shutdown: tokio::sync::watch::Receiver<bool>) {
let (Some(allowed), Some(agent_allow)) =
(env_path(ENV_ALLOWED_FILE), env_path(ENV_AGENT_ALLOW_FILE))
else {
return;
};
let seed = env_path(ENV_SEED_FILE);
let always = std::env::var(ENV_ALWAYS_ALLOWED).unwrap_or_default();
let mut health = SweepHealth::new("squid_allow_render", "warn", FAILURES_BEFORE_WARNING);
// The last `Source` that was logged, so a list that stays unset is
// announced once rather than every pass.
let mut announced: Option<&'static str> = None;
loop {
let files = Files {
allowed: &allowed,
agent_allow: &agent_allow,
};
match pass(&client, &files, seed.as_deref(), &always).await {
Ok(source) => {
health.record_ok();
announce(&source, &mut announced);
}
Err(e) => {
tracing::warn!(error = ?e, "squid allow list: the pass failed");
let err = format!("{e:#}");
health.record_err(|ctx| {
let age = ctx.since_last_ok.map_or_else(
|| "no success this session".to_owned(),
|d| format!("last ok {} ago", sweep_health::fmt_age(d)),
);
format!(
"reading the swarm's egress allow-list is failing ({} consecutive, \
{age}) — squid keeps the list last written: {err}",
ctx.consecutive
)
});
}
}
tokio::select! {
() = tokio::time::sleep(INTERVAL) => {}
_ = shutdown.changed() => return,
}
}
}
/// Log the first pass of a run that left the seed or the kept files.
fn announce(source: &Source, announced: &mut Option<&'static str>) {
let (key, message) = match source {
Source::Swarm => {
*announced = None;
return;
}
Source::Seed => (
"seed",
"squid allow list: the swarm has no list, so squid keeps this hive's own names",
),
Source::Kept => (
"kept",
"squid allow list: the swarm's list is not readable now, so squid keeps the last one written",
),
};
if *announced != Some(key) {
tracing::info!("{message}");
*announced = Some(key);
}
}
async fn pass(
client: &async_nats::Client,
files: &Files<'_>,
seed: Option<&Path>,
always: &str,
) -> Result<Source> {
// An unconnected client does not fail a JetStream request, it hangs on it.
swarm_queue_client::ensure_connected(client)?;
let agents: Vec<String> = crate::lifecycle::agents_for_meta_listing()
.await
.context("enumerating this hive's agents")?
.into_iter()
.map(|spec| spec.name)
.collect();
let read = egress_allow::read(client)
.await
.map_err(|e| anyhow::anyhow!(e))
.context("reading the swarm's egress allow-list")?;
sync_with(read, &agents, seed, always, files)
}
/// [`pass`] with the read, the agents and the files given. Each file is only
/// written when its rendering differs from what it holds.
fn sync_with(
read: Read,
agents: &[String],
seed: Option<&Path>,
always: &str,
files: &Files<'_>,
) -> Result<Source> {
let always: Vec<&str> = always.split_whitespace().collect();
let rendered = match read {
Read::Current(list) => render(&list, agents, &always),
Read::NotSet => {
if super::squid_users::read_or_empty(files.allowed)?.starts_with(SWARM_MARKER) {
return Ok(Source::Kept);
}
seeded(seed)?
}
};
let changed_allowed = write_if_changed(files.allowed, &rendered.allowed)?;
let changed_agents = write_if_changed(files.agent_allow, &rendered.agent_allow)?;
if changed_allowed || changed_agents {
if let Some(counts) = &rendered.counts {
tracing::info!(file = %files.allowed.display(), "{}", counts.summary());
if !counts.skipped.is_empty() {
tracing::warn!(
skipped = ?counts.skipped,
"squid allow list: entries the swarm holds were skipped as invalid"
);
}
} else {
tracing::info!(
file = %files.allowed.display(),
"squid allow list: wrote this hive's own names, no swarm list"
);
}
}
Ok(if rendered.counts.is_some() {
Source::Swarm
} else {
Source::Seed
})
}
/// What the swarm's list rendered as.
struct Counts {
/// Swarm-wide names written.
swarm: usize,
/// Names written across this hive's agents.
agent: usize,
/// Entries left out because they are not valid names or agents.
skipped: Vec<String>,
}
impl Counts {
/// The info line a pass that changed a file logs.
fn summary(&self) -> String {
format!(
"squid allow list: {} swarm, {} agent entries",
self.swarm, self.agent
)
}
}
struct Rendered {
allowed: String,
agent_allow: String,
/// `None` for the seed.
counts: Option<Counts>,
}
/// The files for the seed: its names as they are, and no per-agent rules.
fn seeded(seed: Option<&Path>) -> Result<Rendered> {
let names = match seed {
Some(path) => super::squid_users::read_or_empty(path)?,
None => String::new(),
};
let mut allowed = SEED_MARKER.to_owned();
for line in names.lines().map(str::trim) {
if !line.is_empty() && !line.starts_with('#') {
allowed.push_str(line);
allowed.push('\n');
}
}
Ok(Rendered {
allowed,
agent_allow: SEED_MARKER.to_owned(),
counts: None,
})
}
/// `names` sorted, with each one that another covers dropped.
fn reduced(mut names: Vec<String>) -> Vec<String> {
names.sort();
names.dedup();
let all = names.clone();
names.retain(|n| {
let others: Vec<&String> = all.iter().filter(|o| *o != n).collect();
!covered_by(n, &others)
});
names
}
/// The files for `list`, for the agents this hive runs.
fn render(list: &AllowList, agents: &[String], always: &[&str]) -> Rendered {
let mut skipped = Vec::new();
let mut swarm_wide = Vec::new();
let mut per_agent: std::collections::BTreeMap<&str, Vec<String>> =
std::collections::BTreeMap::new();
for entry in &list.entries {
let Ok(domain) = normalize_domain(&entry.domain) else {
skipped.push(entry.domain.clone());
continue;
};
match entry.agent.as_deref() {
None => swarm_wide.push(domain),
Some(agent) => {
if hive_types::Ident::parse(agent).is_err() {
skipped.push(format!("{} for {agent}", entry.domain));
} else if let Some(mine) = agents.iter().find(|a| *a == agent) {
per_agent.entry(mine.as_str()).or_default().push(domain);
}
}
}
}
swarm_wide.retain(|d| !covered_by(d, always));
let swarm_wide = reduced(swarm_wide);
let mut allowed = SWARM_MARKER.to_owned();
for name in &swarm_wide {
allowed.push_str(name);
allowed.push('\n');
}
let mut agent_rules = Vec::new();
let mut agent_count = 0;
for (agent, mut names) in per_agent {
names.retain(|d| !covered_by(d, always) && !covered_by(d, &swarm_wide));
let names = reduced(names);
if names.is_empty() {
continue;
}
agent_count += names.len();
agent_rules.push(format!(
"acl agent_allow_{agent} proxy_auth {agent}\n\
acl agent_allow_{agent}_dst dstdomain {}\n\
http_access allow agent_allow_{agent} agent_allow_{agent}_dst\n",
names.join(" ")
));
}
Rendered {
allowed,
agent_allow: format!("{SWARM_MARKER}{}", agent_rules.concat()),
counts: Some(Counts {
swarm: swarm_wide.len(),
agent: agent_count,
skipped,
}),
}
}
#[cfg(test)]
mod tests {
use super::*;
use swarm_wire::egress_allow::Entry;
fn entry(domain: &str, agent: Option<&str>) -> Entry {
Entry {
domain: domain.to_owned(),
agent: agent.map(str::to_owned),
}
}
fn list(entries: &[(&str, Option<&str>)]) -> AllowList {
AllowList {
entries: entries.iter().map(|(d, a)| entry(d, *a)).collect(),
}
}
fn agents(names: &[&str]) -> Vec<String> {
names.iter().map(|n| (*n).to_owned()).collect()
}
const ALWAYS: &[&str] = &["api.anthropic.com", ".claude.ai", "cache.nixos.org"];
fn rendered(entries: &[(&str, Option<&str>)], mine: &[&str]) -> Rendered {
render(&list(entries), &agents(mine), ALWAYS)
}
#[test]
fn swarm_wide_names_are_sorted_one_per_line_after_the_marker() {
let r = rendered(
&[
("github.com", None),
("crates.io", None),
("GitHub.com", None),
],
&["atlas"],
);
assert_eq!(r.allowed, "# source: swarm\ncrates.io\ngithub.com\n");
let counts = r.counts.expect("a swarm list");
assert_eq!((counts.swarm, counts.agent), (2, 0));
}
#[test]
fn an_empty_list_renders_an_empty_set_of_names_and_no_rules() {
let r = rendered(&[], &["atlas"]);
assert_eq!(r.allowed, SWARM_MARKER);
assert_eq!(r.agent_allow, SWARM_MARKER);
let counts = r.counts.expect("an empty swarm list is still a swarm list");
assert_eq!((counts.swarm, counts.agent), (0, 0));
}
#[test]
fn an_agents_names_become_a_user_rule_and_other_hives_agents_are_left_out() {
let r = rendered(
&[
("pypi.org", Some("atlas")),
("crates.io", Some("atlas")),
("github.com", Some("iris")),
("github.com", Some("../argus")),
],
&["atlas", "argus"],
);
assert_eq!(
r.agent_allow,
"# source: swarm\n\
acl agent_allow_atlas proxy_auth atlas\n\
acl agent_allow_atlas_dst dstdomain crates.io pypi.org\n\
http_access allow agent_allow_atlas agent_allow_atlas_dst\n"
);
let counts = r.counts.expect("a swarm list");
assert_eq!((counts.swarm, counts.agent), (0, 2));
// The name that could break out of the line is reported, not rendered.
assert_eq!(counts.skipped, ["github.com for ../argus"]);
assert!(!r.agent_allow.contains("iris"), "another hive's agent");
assert!(!r.agent_allow.contains("argus"), "an invalid name");
}
#[test]
fn an_entry_that_is_not_a_name_never_reaches_squid_configuration() {
let r = rendered(
&[
("github.com", None),
("1.2.3.4", None),
("*.example.com", Some("atlas")),
("example.com\nhttp_access allow all", Some("atlas")),
("example.com\nhttp_access allow all", None),
],
&["atlas"],
);
assert_eq!(r.allowed, "# source: swarm\ngithub.com\n");
assert_eq!(r.agent_allow, SWARM_MARKER);
assert!(!r.allowed.contains("http_access") && !r.agent_allow.contains("http_access"));
assert_eq!(r.counts.expect("a swarm list").skipped.len(), 4);
}
#[test]
fn a_name_squid_allows_everyone_or_the_swarm_already_lists_is_dropped() {
let r = rendered(
&[
("api.anthropic.com", None),
("a.claude.ai", None),
("cache.nixos.org", Some("atlas")),
(".github.com", None),
("api.github.com", None),
("api.github.com", Some("atlas")),
("crates.io", Some("atlas")),
],
&["atlas"],
);
assert_eq!(r.allowed, "# source: swarm\n.github.com\n");
assert_eq!(
r.agent_allow,
"# source: swarm\n\
acl agent_allow_atlas proxy_auth atlas\n\
acl agent_allow_atlas_dst dstdomain crates.io\n\
http_access allow agent_allow_atlas agent_allow_atlas_dst\n"
);
// Control: the same entries survive when nothing is always allowed.
let bare = render(
&list(&[("api.anthropic.com", None), ("a.claude.ai", None)]),
&agents(&["atlas"]),
&[],
);
assert_eq!(
bare.allowed,
"# source: swarm\na.claude.ai\napi.anthropic.com\n"
);
}
#[test]
fn the_log_line_carries_the_counts() {
let r = rendered(
&[
("github.com", None),
("crates.io", None),
("pypi.org", Some("atlas")),
("npmjs.org", Some("atlas")),
("docs.rs", Some("argus")),
],
&["atlas", "argus"],
);
let counts = r.counts.expect("a swarm list");
assert_eq!(
counts.summary(),
"squid allow list: 2 swarm, 3 agent entries"
);
}
struct Dir(tempfile::TempDir);
impl Dir {
fn allowed(&self) -> PathBuf {
self.0.path().join("allowed")
}
fn agent_allow(&self) -> PathBuf {
self.0.path().join("agent-allow.conf")
}
fn seed(&self) -> PathBuf {
self.0.path().join("seed")
}
}
fn dir(seed: &str) -> Dir {
let dir = Dir(tempfile::tempdir().expect("tempdir"));
std::fs::write(dir.seed(), seed).expect("seed");
dir
}
fn sync(d: &Dir, read: Read) -> Result<Source> {
sync_with(
read,
&agents(&["atlas"]),
Some(&d.seed()),
"api.anthropic.com cache.nixos.org",
&Files {
allowed: &d.allowed(),
agent_allow: &d.agent_allow(),
},
)
}
fn text(path: &Path) -> String {
std::fs::read_to_string(path).expect("the file is there")
}
#[test]
fn without_a_swarm_list_the_files_hold_the_seed() {
let d = dir("# a comment\nforge.example\n\n loop.example \n");
assert_eq!(sync(&d, Read::NotSet).expect("renders"), Source::Seed);
assert_eq!(
text(&d.allowed()),
"# source: seed\nforge.example\nloop.example\n"
);
assert_eq!(text(&d.agent_allow()), SEED_MARKER);
// A second pass writes nothing.
let ino = std::os::unix::fs::MetadataExt::ino(&std::fs::metadata(d.allowed()).unwrap());
sync(&d, Read::NotSet).expect("renders");
assert_eq!(
std::os::unix::fs::MetadataExt::ino(&std::fs::metadata(d.allowed()).unwrap()),
ino
);
}
#[test]
fn a_swarm_list_replaces_the_seed_even_when_empty() {
let d = dir("forge.example\n");
sync(&d, Read::NotSet).expect("seeded");
assert!(text(&d.allowed()).contains("forge.example"), "control");
let empty = Read::Current(AllowList::default());
assert_eq!(sync(&d, empty).expect("renders"), Source::Swarm);
assert_eq!(text(&d.allowed()), SWARM_MARKER);
let named = Read::Current(list(&[("github.com", None), ("pypi.org", Some("atlas"))]));
sync(&d, named).expect("renders");
assert_eq!(text(&d.allowed()), "# source: swarm\ngithub.com\n");
assert!(text(&d.agent_allow()).contains("acl agent_allow_atlas proxy_auth atlas"));
}
#[test]
fn a_swarm_list_that_stops_answering_is_kept_and_a_seed_is_refreshed() {
let d = dir("forge.example\n");
sync(&d, Read::Current(list(&[("github.com", None)]))).expect("renders");
let before = (text(&d.allowed()), text(&d.agent_allow()));
assert_eq!(sync(&d, Read::NotSet).expect("renders"), Source::Kept);
assert_eq!((text(&d.allowed()), text(&d.agent_allow())), before);
// Control: a file that never held the swarm's list follows the seed.
let d = dir("forge.example\n");
std::fs::write(d.allowed(), "old.example\n").expect("a tmpfiles-seeded file");
assert_eq!(sync(&d, Read::NotSet).expect("renders"), Source::Seed);
assert_eq!(text(&d.allowed()), "# source: seed\nforge.example\n");
}
#[test]
fn announce_logs_a_run_of_unset_passes_once() {
let mut announced = None;
announce(&Source::Seed, &mut announced);
assert_eq!(announced, Some("seed"));
announce(&Source::Swarm, &mut announced);
assert_eq!(announced, None, "a list that appears resets the run");
announce(&Source::Kept, &mut announced);
assert_eq!(announced, Some("kept"));
}
#[test]
fn the_files_are_group_readable() {
use std::os::unix::fs::PermissionsExt as _;
let d = dir("");
sync(&d, Read::Current(list(&[("github.com", None)]))).expect("renders");
for file in [d.allowed(), d.agent_allow()] {
let mode = std::fs::metadata(file).expect("stat").permissions().mode() & 0o777;
assert_eq!(mode, 0o640);
}
}
}

4
hive-c0re/src/workers/squid_users.rs
View file

@ -233,7 +233,7 @@ where
}
/// The file's text, or the empty string when it does not exist.
fn read_or_empty(file: &Path) -> Result<String> {
pub(super) fn read_or_empty(file: &Path) -> Result<String> {
match std::fs::read_to_string(file) {
Ok(text) => Ok(text),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(String::new()),
@ -243,7 +243,7 @@ fn read_or_empty(file: &Path) -> Result<String> {
/// Write `text` to `file` unless it already holds exactly that. Returns
/// whether it wrote.
fn write_if_changed(file: &Path, text: &str) -> Result<bool> {
pub(super) fn write_if_changed(file: &Path, text: &str) -> Result<bool> {
if read_or_empty(file)? == text {
return Ok(false);
}

7
nix/host-modules/hive-c0re/environment.nix
View file

@ -301,6 +301,13 @@ in
# store under the identity above.
HIVE_SQUID_USERS_FILE = config.services.hyperhive.squid.usersFile;
HIVE_SQUID_UNRESTRICTED_FILE = config.services.hyperhive.squid.unrestrictedFile;
# The swarm's egress allow-list, which `workers::squid_allow` renders into
# the two files; the seed is the list the files hold until the swarm has one,
# and the always-allowed names are those it never repeats.
HIVE_SQUID_ALLOWED_FILE = config.services.hyperhive.squid.allowedFile;
HIVE_SQUID_AGENT_ALLOW_FILE = config.services.hyperhive.squid.agentAllowFile;
HIVE_SQUID_ALLOWED_SEED_FILE = config.services.hyperhive.squid.allowedSeedFile;
HIVE_SQUID_ALWAYS_ALLOWED = lib.concatStringsSep " " config.services.hyperhive.squid.alwaysAllowed;
# The egress proxy facts every container is handed (`HiveFacts` in
# `hive-priv-sock`). They sit with the users file above because that is what
# makes squid's credential check and per-agent policy real on this host: an

154
nix/host-modules/hive-squid.nix
View file

@ -4,7 +4,10 @@
# must carry a per-agent proxy credential, checked against the users file
# hive-c0re renders. A credential's agent either has the swarm's
# `unrestricted_internet` flag, and may then reach any destination the deny
# rules leave, or reaches only the listed names. An agent sends its credential
# rules leave, or reaches only the listed names: the base list, the allow-list
# the swarm keeps (rendered by hive-c0re beside the users file, with this
# host's `allowedDomains` as the list until the swarm has one) and the names
# the swarm lists for that agent alone. An agent sends its credential
# once it has fetched it (../agent-modules/egress-proxy.nix); the exporter,
# journal forwarding and the self-check below show the proxy works.
{
@ -26,6 +29,16 @@ let
allowed = lib.unique (cfg.allowedDomains ++ [ cfg.selfcheck.canary ]);
base = lib.unique cfg.baseAllowedDomains;
# The names squid allows every authenticated user whatever the swarm lists.
# hive-c0re skips an allow-list entry that one of them already covers.
alwaysAllowed = base ++ [ cfg.selfcheck.canary ];
# What `allowedFile` holds until the swarm has an allow-list: this hive's
# own `allowedDomains`, one per line. The canary is in the configuration.
seedFile = pkgs.writeText "squid-allowed-seed" (
lib.concatMapStrings (d: d + "\n") (
lib.unique (lib.subtractLists [ cfg.selfcheck.canary ] cfg.allowedDomains)
)
);
# The squid user the self-check logs in as: `swarm_secret_client::proxy::SELFCHECK_USER`.
selfcheckUser = "selfcheck.squid";
@ -65,8 +78,10 @@ let
# denies come first because `dstdomain` on an IP literal does a reverse
# lookup. The credential check sits right before the first allow, so only a
# request that would otherwise be let through gets the 407 challenge. Every
# user is then let through to the base names, an unrestricted user to any other
# destination on a safe port, and every other user to the listed names. The `dst` denies cost an
# user is then let through to the base names, a user the swarm lists names
# for to those names (the include, whose rules hive-c0re generates and whose
# file may be empty), an unrestricted user to any other destination on a
# safe port, and every other user to the listed names. The `dst` denies cost an
# unlisted name a lookup on the host's resolver before it is refused.
squidConf = ''
http_port ${listen}
@ -107,9 +122,8 @@ let
acl ip_v4_host url_regex ^([a-z]+://)?[0-9.]+(:[0-9]+)?(/|$)
acl ip_v6_host url_regex ^([a-z]+://)?\[[0-9a-fA-F:.]+\](:[0-9]+)?(/|$)
${lib.optionalString (base != [ ]) "acl base_dst dstdomain ${lib.concatStringsSep " " base}"}
${lib.optionalString (
allowed != [ ]
) "acl allowed_dst dstdomain ${lib.concatStringsSep " " allowed}"}
acl allowed_dst dstdomain ${cfg.selfcheck.canary}
acl allowed_dst dstdomain "${cfg.allowedFile}"
# Answered by squid itself, never forwarded, and addressed to the bridge IP
# on a port outside Safe_ports, so it precedes the destination rules.
@ -126,12 +140,30 @@ let
http_access deny to_private
http_access deny !authed
${lib.optionalString (base != [ ]) "http_access allow base_dst"}
include ${cfg.agentAllowFile}
http_access allow unrestricted
${lib.optionalString (allowed != [ ]) "http_access deny !allowed_dst"}
${lib.optionalString (allowed != [ ]) "http_access allow allowed_dst"}
http_access deny !allowed_dst
http_access allow allowed_dst
http_access deny all
'';
# `squid -k parse` on the configuration with the include pointing at an empty
# file. The nixpkgs module runs the same parse on the real text, which
# cannot open the include in the build sandbox.
squidConfCheck =
pkgs.runCommand "squid-conf-check"
{
nativeBuildInputs = [ config.services.squid.package ];
}
''
squid -k parse -f ${
pkgs.writeText "squid.conf" (
builtins.replaceStrings [ cfg.agentAllowFile ] [ "/dev/null" ] squidConf
)
}
touch $out
'';
sandbox = {
NoNewPrivileges = true;
ProtectSystem = "strict";
@ -222,8 +254,14 @@ in
"crates.io"
];
description = ''
Hostnames squid lets a client reach, by CONNECT or plain HTTP. Each
entry matches that exact name. Everything else gets a 403, except
Hostnames squid lets a client reach, by CONNECT or plain HTTP, until the
swarm has an allow-list of its own: the list the swarm's controller
keeps (`/api/egress/allowed-domains`) replaces this one as soon as it
holds an entry or has had one, and this option is the list again only
while the controller has none, for instance because it predates the
feature. hive-c0re writes whichever list applies to `allowedFile`.
Each entry matches that exact name, or that name and its subdomains
with a leading dot. Everything else gets a 403, except
for an unrestricted user. The swarm's own service names are not
listed: they resolve to the bridge or a private address, which the
destination rules refuse, and agents reach them directly. The
@ -238,11 +276,7 @@ in
baseAllowedDomains = lib.mkOption {
type = lib.types.listOf lib.types.str;
default = [
"api.anthropic.com"
"claude.ai"
"platform.claude.com"
];
default = import ./lib/squid-base-domains.nix;
example = [
"api.anthropic.com"
"claude.ai"
@ -261,7 +295,9 @@ in
environment, which this host does not read. Swarm-internal names are
not listed, since agents reach them directly. The destination denies
still apply to every entry, and the list is separate from
`allowedDomains`.
`allowedDomains`. A name the swarm's allow-list holds that this list
already covers is left out of the rendered files, and the controller
refuses it as an entry when it is among the defaults.
'';
};
@ -289,12 +325,66 @@ in
`unrestricted_internet` flag (set unless the swarm records it off), and
the self-check's unrestricted user. hive-c0re renders it beside
`usersFile`, and squid is reloaded when it changes. Every other user
reaches only `allowedDomains`. An empty file, which is what exists
reaches only the base list, the allow-list and its own names. An empty file, which is what exists
until hive-c0re writes one, makes every user a restricted one.
Owned by `hive-core` and readable by the `squid` group.
'';
};
allowedFile = lib.mkOption {
type = lib.types.path;
default = "${builtins.dirOf cfg.usersFile}/allowed";
readOnly = true;
defaultText = lib.literalMD "`allowed`, in the directory of `usersFile`";
description = ''
The names squid lets every restricted user reach, one per line, read as
part of its allow ACL beside the self-check canary. It starts as a copy
of `allowedDomains`. hive-c0re rewrites it every pass: with the swarm's
swarm-wide entries once the controller has a list, otherwise with
`allowedDomains`. Squid is reloaded when it changes. Owned by
`hive-core` and readable by the `squid` group.
'';
};
agentAllowFile = lib.mkOption {
type = lib.types.path;
default = "${builtins.dirOf cfg.usersFile}/agent-allow.conf";
readOnly = true;
defaultText = lib.literalMD "`agent-allow.conf`, in the directory of `usersFile`";
description = ''
Squid configuration that hive-c0re generates and the main configuration
includes after every destination deny and `deny !authed`, where the
base list is allowed: for each agent of this hive the swarm lists
names for, an ACL for its user, an ACL for its names and the
`http_access allow` joining the two. Empty until the controller has a
list. Squid is reloaded when it changes. Owned by `hive-core` and
readable by the `squid` group.
'';
};
allowedSeedFile = lib.mkOption {
type = lib.types.path;
default = seedFile;
readOnly = true;
defaultText = lib.literalMD "`allowedDomains`, one per line, minus the self-check canary";
description = ''
The names `allowedFile` starts as and hive-c0re writes to it again while
the swarm has no allow-list. Read-only.
'';
};
alwaysAllowed = lib.mkOption {
type = lib.types.listOf lib.types.str;
default = alwaysAllowed;
readOnly = true;
defaultText = lib.literalMD "`baseAllowedDomains` and the self-check canary";
description = ''
The names squid allows every authenticated user whatever the swarm
lists. hive-c0re leaves a swarm allow-list entry out of the rendered
files when one of these covers it. Read-only.
'';
};
selfcheck.credentialName = lib.mkOption {
type = lib.types.str;
default = "hive-squid.selfcheck";
@ -348,31 +438,45 @@ in
services.squid = {
enable = true;
configText = squidConf;
# The module's build-time parse fails on the `include` of a file only the
# running host has; `squidConfCheck` parses the same text without it.
validateConfig = false;
};
system.extraDependencies = [ squidConfCheck ];
# The directory is setgid so the file hive-c0re renames into it is the
# squid group's. The users file exists before squid starts, empty: the
# helper exits on a missing file and squid then rejects every login. The
# unrestricted list is created the same way, though squid treats a missing
# one as empty.
# one as empty. The allow-list file starts as a copy of `allowedDomains`
# and an existing one is left alone, so the list hive-c0re wrote survives a
# switch. The include is empty until hive-c0re has a list: squid does not
# start on a missing one.
systemd.tmpfiles.rules = [
"d ${usersDir} 2750 hive-core squid - -"
"f ${cfg.usersFile} 0640 hive-core squid - -"
"f ${cfg.unrestrictedFile} 0640 hive-core squid - -"
"C ${cfg.allowedFile} 0640 hive-core squid - ${seedFile}"
"f ${cfg.agentAllowFile} 0640 hive-core squid - -"
];
# Squid reads the unrestricted list when it parses its configuration, not
# when it changes. hive-c0re writes the list only when its contents change,
# so each change is one reload. Nothing happens to a squid that is not
# running; it reads the list when it starts.
# Squid reads the unrestricted list, the allow-list and the per-agent include
# when it parses its configuration, not when they change. hive-c0re writes
# each only when its contents change, so each change is one reload. Nothing
# happens to a squid that is not running; it reads them when it starts.
systemd.paths.squid-unrestricted-reload = {
description = "Watch squid's unrestricted-users list";
description = "Watch the lists hive-c0re renders for squid";
wantedBy = [ "paths.target" ];
pathConfig.PathChanged = cfg.unrestrictedFile;
pathConfig.PathChanged = [
cfg.unrestrictedFile
cfg.allowedFile
cfg.agentAllowFile
];
};
systemd.services.squid-unrestricted-reload = {
description = "Reload squid to read its unrestricted-users list";
description = "Reload squid to read the lists hive-c0re renders";
# Several list writes in a few seconds must not trip the start limit: a
# tripped limit also fails the path unit, and later writes then reload nothing.
startLimitIntervalSec = 0;

9
nix/host-modules/lib/squid-base-domains.nix Normal file
View file

@ -0,0 +1,9 @@
# The default of `services.hyperhive.squid.baseAllowedDomains`, and the names
# swarm-controller refuses as allow-list entries because squid already allows
# them to every authenticated user: the hosts Claude Code requires
# (<https://code.claude.com/docs/en/network-config>).
[
"api.anthropic.com"
"claude.ai"
"platform.claude.com"
]

8
nix/host-modules/swarm-controller.nix
View file

@ -864,6 +864,14 @@ in
# token is in the store, written by `swarm-bao-grafana-controller-token`
# on Grafana's host.
SWARM_CONTROLLER_GRAFANA_URL = "https://${swarmCfg.grafana.domain}";
# The names squid allows every authenticated agent by default. The
# allow-list route refuses them as entries
# (swarm-controller/src/egress_allow.rs). A hive that overrides
# `squid.baseAllowedDomains` is not reflected here; its hive-c0re
# skips the entries its own list covers.
SWARM_CONTROLLER_SQUID_BASE_DOMAINS = lib.concatStringsSep " " (
import ./lib/squid-base-domains.nix
);
}
// queueEnv
// forgeEnv

14
nix/module-eval/bao-controller.nix
View file

@ -249,6 +249,20 @@ let
controllerNoStore
];
}
{
# The allow-list route refuses the names squid already allows everyone.
# The list the controller is handed is the hive module's default, from
# one file; the second clause is the control that the default is not empty.
name = "the controller is handed the base list squid defaults to";
ok =
let
base = baoControllerHere.services.hyperhive.squid.baseAllowedDomains;
in
base != [ ]
&&
baoControllerHere.systemd.services.swarm-controller.environment.SWARM_CONTROLLER_SQUID_BASE_DOMAINS
== lib.concatStringsSep " " base;
}
{
# A hive's cert-auth role carries the authority by value, so the daemon
# has to be handed the file rather than a path into the store's own

58
nix/module-eval/hive-otel.nix
View file

@ -47,6 +47,12 @@ let
hiveOtelAllLocal = hiveWithController {
deploy.bao.enable = true;
deploy.authelia.enable = true;
# The seed `allowedFile` starts as; the canary is listed too, to show it
# is not written twice.
squid.allowedDomains = [
"github.com"
"cache.nixos.org"
];
};
# Neither a leaf nor a secret file: the one shape the collector has no
@ -500,12 +506,18 @@ let
base = at "http_access allow base_dst";
unrestricted = at "http_access allow unrestricted";
authed = at "http_access deny !authed";
firstAllow = lib.min base (lib.min unrestricted allow);
# The generated per-agent `http_access allow` rules sit in this file.
agentRules = at "include ${hiveOtel.services.hyperhive.squid.agentAllowFile}";
firstAllow = lib.min agentRules (lib.min base (lib.min unrestricted allow));
in
allow != null
&& base != null
&& unrestricted != null
&& authed != null
&& agentRules != null
&& authed < agentRules
&& base < agentRules
&& agentRules < unrestricted
&& at "http_access deny to_localhost" != null
&& at "http_access deny to_linklocal" != null
&& at "http_access deny to_bridge" != null
@ -602,13 +614,55 @@ let
path = hiveOtel.systemd.paths.squid-unrestricted-reload;
svc = hiveOtel.systemd.services.squid-unrestricted-reload.serviceConfig;
in
path.pathConfig.PathChanged == list
path.pathConfig.PathChanged == [
list
hiveOtel.services.hyperhive.squid.allowedFile
hiveOtel.services.hyperhive.squid.agentAllowFile
]
&& hiveOtel.systemd.services.squid-unrestricted-reload.startLimitIntervalSec == 0
&& builtins.elem "paths.target" path.wantedBy
&& lib.hasSuffix "systemctl try-reload-or-restart squid.service" svc.ExecStart
&& builtins.elem "f ${list} 0640 hive-core squid - -" rules
&& hiveOtelAllLocal.systemd.services.hive-c0re.environment.HIVE_SQUID_UNRESTRICTED_FILE == list;
}
{
# The swarm's allow-list reaches squid as a file of names, which starts
# as this hive's own `allowedDomains` (the canary left out, since the
# configuration names it), and as an include that holds the per-agent
# rules. Control: the seed holds the listed name, and the configuration
# no longer names it, so the absence of the canary and of the list from
# the seed and the configuration is not a lookup of nothing.
name = "squid reads the swarm's allow-list from a file seeded with the hive's own names";
ok =
let
sq = hiveOtelAllLocal.services.hyperhive.squid;
lines = lib.splitString "\n" hiveOtelAllLocal.services.squid.configText;
rules = hiveOtelAllLocal.systemd.tmpfiles.rules;
env = hiveOtelAllLocal.systemd.services.hive-c0re.environment;
in
builtins.elem ''acl allowed_dst dstdomain "${sq.allowedFile}"'' lines
&& builtins.elem "acl allowed_dst dstdomain ${sq.selfcheck.canary}" lines
&& !(lib.hasInfix "github.com" hiveOtelAllLocal.services.squid.configText)
&& sq.allowedFile == "/var/lib/hive-squid/allowed"
&& sq.agentAllowFile == "/var/lib/hive-squid/agent-allow.conf"
&& sq.allowedSeedFile.text == "github.com\n"
&& builtins.elem "C ${sq.allowedFile} 0640 hive-core squid - ${sq.allowedSeedFile}" rules
&& builtins.elem "f ${sq.agentAllowFile} 0640 hive-core squid - -" rules
&& env.HIVE_SQUID_ALLOWED_FILE == sq.allowedFile
&& env.HIVE_SQUID_AGENT_ALLOW_FILE == sq.agentAllowFile
&& env.HIVE_SQUID_ALLOWED_SEED_FILE == toString sq.allowedSeedFile
&&
env.HIVE_SQUID_ALWAYS_ALLOWED == "api.anthropic.com claude.ai platform.claude.com cache.nixos.org";
}
{
# The nixpkgs module parses the configuration at build time, which cannot
# open the include; the parse moves to a check on the same text, so a
# broken configuration still fails the host build.
name = "squid's configuration is still parsed at build time, without the include";
ok =
!hiveOtel.services.squid.validateConfig
&& lib.any (d: (d.name or "") == "squid-conf-check") hiveOtel.system.extraDependencies;
}
{
# The helper is the NCSA one, on the file hive-c0re renders. The file
# exists, empty, before squid starts (the helper exits on a missing one),

45
nix/vm-tests/hive-squid.nix
View file

@ -152,6 +152,8 @@ pkgs.testers.runNixOSTest {
"cache.t.local"
"api.t.local"
"other.t.local"
"swarm.t.local"
"agentonly.t.local"
"deny-canary.invalid"
];
"127.0.0.1" = [ "loop.t.local" ];
@ -231,6 +233,7 @@ pkgs.testers.runNixOSTest {
testScript = ''
import re
import secrets
import shlex
import time
# Drawn fresh each run. The users file holds only their hashes, as
@ -248,6 +251,8 @@ pkgs.testers.runNixOSTest {
RUTH = "ruth:" + PW["ruth"]
USERS = "/var/lib/hive-squid/users"
LIST = "/var/lib/hive-squid/unrestricted"
ALLOWED = "/var/lib/hive-squid/allowed"
AGENT_ALLOW = "/var/lib/hive-squid/agent-allow.conf"
UNRESTRICTED = ["uma", "selfcheck-unrestricted.squid"]
start_all()
@ -289,6 +294,16 @@ pkgs.testers.runNixOSTest {
f"&& chmod 640 {LIST}.new && mv {LIST}.new {LIST}"
)
def write_allow(allowed, agent_conf):
# What hive-c0re does for its two files: written aside and renamed into
# place, in squid's group. Squid reads them on a reload, which
# squid-unrestricted-reload does when either changes.
for path, body in ((ALLOWED, allowed), (AGENT_ALLOW, agent_conf)):
proxy.succeed(
f"printf '%s' {shlex.quote(body)} > {path}.new && chown hive-core:squid {path}.new "
f"&& chmod 640 {path}.new && mv {path}.new {path}"
)
def wait_status(host, cred, want):
# The reload that follows a list write is asynchronous.
got = None
@ -310,6 +325,11 @@ pkgs.testers.runNixOSTest {
assert proxy.succeed("stat -c %a /var/lib/hive-squid").strip() == "2750"
assert proxy.succeed(f"stat -c %s {USERS}").strip() == "0"
assert proxy.succeed(f"stat -c %s {LIST}").strip() == "0"
assert proxy.succeed(f"stat -c '%U:%G %a' {ALLOWED}").strip() == "hive-core:squid 640"
assert proxy.succeed(f"stat -c '%U:%G %a' {AGENT_ALLOW}").strip() == "hive-core:squid 640"
assert proxy.succeed(f"stat -c %s {AGENT_ALLOW}").strip() == "0"
# The allow-list file starts as the hive's own allowedDomains.
proxy.succeed(f"grep -Fx forge.t.local {ALLOWED}")
assert connect("forge.t.local", ATLAS) == "407", "no such user"
assert connect("other.t.local", UMA) == "407", "no such user, unlisted name"
@ -500,6 +520,31 @@ pkgs.testers.runNixOSTest {
wait_status("other.t.local", RUTH, "403")
assert connect("other.t.local", UMA) == "200", "control: uma is still listed"
with subtest("the swarm's allow-list replaces the hive's names, opens one to every restricted user, and a per-agent rule opens one to that agent alone"):
seeded = proxy.succeed(f"cat {ALLOWED}")
assert connect("swarm.t.local", RUTH) == "403", "control: not allowed yet"
assert connect("agentonly.t.local", ATLAS) == "403", "control: not allowed yet"
assert connect("forge.t.local", RUTH) == "200", "control: the hive's own name"
write_allow(
"# source: swarm\nswarm.t.local\n",
"# source: swarm\n"
"acl agent_allow_atlas proxy_auth atlas\n"
"acl agent_allow_atlas_dst dstdomain agentonly.t.local priv10.t.local\n"
"http_access allow agent_allow_atlas agent_allow_atlas_dst\n",
)
wait_status("swarm.t.local", RUTH, "200")
wait_status("agentonly.t.local", ATLAS, "200")
assert connect("swarm.t.local", ATLAS) == "200", "swarm-wide, another user"
assert connect("agentonly.t.local", RUTH) == "403", "the per-agent name, another restricted user"
assert connect("agentonly.t.local") == "407", "no credential"
assert connect("forge.t.local", RUTH) == "403", "the hive's own name is replaced"
assert connect("priv10.t.local", ATLAS) == "403", "a per-agent name that resolves privately"
assert connect("agentonly.t.local", UMA) == "200", "control: unrestricted reaches it anyway"
write_allow(seeded, "")
wait_status("swarm.t.local", RUTH, "403")
wait_status("agentonly.t.local", ATLAS, "403")
assert connect("forge.t.local", RUTH) == "200", "the hive's own name is back"
with subtest("squid starts on a missing list, treats every user as restricted, and recovers when it appears"):
proxy.succeed(f"rm {LIST}")
proxy.succeed("systemctl restart squid.service")

56
swarm-controller-api/src/egress.rs Normal file
View file

@ -0,0 +1,56 @@
//! `/api/egress/allowed-domains` — the destinations the swarm lets agents
//! reach through their hive's egress proxy.
use serde::{Deserialize, Serialize};
/// One allowed destination and who may reach it.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "schema", derive(utoipa::ToSchema))]
#[serde(deny_unknown_fields)]
pub struct AllowedDomain {
/// A DNS name of at least two labels, such as `github.com`, stored
/// lower-cased. One leading dot, as in `.githubusercontent.com`, also
/// allows every subdomain. IP addresses, wildcards, ports, paths and the
/// proxy's own always-allowed names are refused.
#[cfg_attr(feature = "schema", schema(example = "github.com"))]
pub domain: String,
/// The agent this entry is for. Absent means every agent in the swarm.
#[serde(default, skip_serializing_if = "Option::is_none")]
#[cfg_attr(feature = "schema", schema(example = "atlas"))]
pub agent: Option<String>,
}
/// The swarm's allow-list: every entry, swarm-wide and per agent, sorted.
#[derive(Debug, Default, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "schema", derive(utoipa::ToSchema))]
pub struct AllowedDomains {
pub entries: Vec<AllowedDomain>,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn an_entry_leaves_out_an_unset_agent_and_refuses_unknown_fields() {
let swarm_wide = AllowedDomain {
domain: "github.com".to_owned(),
agent: None,
};
let wire = r#"{"domain":"github.com"}"#;
assert_eq!(serde_json::to_string(&swarm_wide).unwrap(), wire);
assert_eq!(
serde_json::from_str::<AllowedDomain>(wire).unwrap(),
swarm_wide
);
let scoped = r#"{"domain":"crates.io","agent":"atlas"}"#;
assert_eq!(
serde_json::from_str::<AllowedDomain>(scoped)
.unwrap()
.agent
.as_deref(),
Some("atlas")
);
assert!(serde_json::from_str::<AllowedDomain>(r#"{"domain":"a.io","hive":"x"}"#).is_err());
}
}

1
swarm-controller-api/src/lib.rs
View file

@ -6,6 +6,7 @@
pub mod agents;
pub mod agents_md;
pub mod egress;
pub mod forge;
pub mod inference;
pub mod limits;

Some files were not shown because too many files have changed in this diff Show more