always include a hive's own allowedDomains in the rendered swarm-wide list

docs/networking/network.md

@@ -307,8 +307,8 @@ host. It never decrypts TLS, so it holds no signing key. It caches nothing.
   which the rules below refuse, and agents reach them directly. Each entry
   matches that exact name, or with a leading dot the name and its subdomains. A user on the unrestricted list reaches any
   other name too, on ports 80 and 443. Everyone else gets a 403 for it.
-  `allowedDomains` is the hive's own list. Squid uses it until the swarm has a
+  `allowedDomains` is the hive's own list, always allowed whether or not the
-  list of its own (next bullet).
+  swarm has a list of its own (next bullet).
 - **Swarm allow-list.** The operator adds domains in the swarm UI
   ([allowed domains](../swarm/ui.md#allowed-domains)); the swarm controller
   keeps them in one value of the `egress-allow` bucket, each for every agent
@@ -344,11 +344,13 @@ host. It never decrypts TLS, so it holds no signing key. It caches nothing.
   the `opencode` preset's `acp.opencode.provider.baseUrl`, get no derived
   entry, so add one in the swarm UI. A `hive-c0re` that predates the field
   renders a derived entry like any other.
-  The swarm's list replaces the hive's `allowedDomains` as soon as it exists,
+  The swarm's list adds to the hive's `allowedDomains`: `allowedFile` holds
-  even when empty: add the names the hive allowed to the swarm's list
+  those names plus the swarm-wide entries, so removing an entry never removes a
-  before relying on it. Until the controller has a list (it predates the
+  name the hive lists itself. Until the controller has a list (it predates the
   feature, or nobody added an entry) `allowedFile` holds `allowedDomains` and
-  `agent-allow.conf` is empty. A queue that stops answering for the bucket
+  `agent-allow.conf` is empty. A `hive-c0re` older than this rule replaces
+  `allowedDomains` with the swarm's list once the controller has one, so every
+  hive must run the new `hive-c0re` before the controller's first entry exists. A queue that stops answering for the bucket
   leaves a file the swarm wrote untouched, and a failed read leaves both
   files untouched. The first line of a file written from the swarm's list is a
   `# source: swarm` comment. The queue grant a hive needs to read the list

docs/swarm/ui.md

@@ -655,9 +655,8 @@ an entry that exists changes nothing.
 A saved change isn't instant. Each hive reads the list about every 30
 seconds and reloads its proxy, so the name is reachable within a minute. The
 proxy's private-range rules still apply: an entry whose name resolves to a
-loopback, private or bridge address stays blocked. The first entry makes the
+loopback, private or bridge address stays blocked. An entry adds to the names a
-swarm's list replace the names a hive's own `allowedDomains` lists, so add
+hive's own `allowedDomains` lists, which stay allowed. See [the egress
-those names here too; a hive keeps its own list until then. See [the egress
 proxy](../networking/network.md#egress-proxy-squid) for what each hive does
 when the controller is old or unreachable.
 

frontend/packages/swarm-ui/src/agent/detail/AllowedDomains.tsx

@@ -10,9 +10,8 @@
 // allows everyone) shows as it arrives, next to the box. The hive reads a
 // change within about 30 seconds.
 //
-// The first entry makes the swarm's list replace the names the hive's own
+// An entry adds to the names the hive's own configuration allows, which stay
-// configuration allows, so a name the hive allowed and the swarm's list lacks
+// allowed. An agent that holds the unrestricted
-// stops being allowed once hives read it. An agent that holds the unrestricted
 // internet flag reaches any name the proxy's private-range rules leave, so an
 // entry matters only to a restricted one.
 //
@@ -223,9 +222,8 @@ function AddForm({
         such as <code>github.com</code>; a leading dot, as in{" "}
         <code>.githubusercontent.com</code>, also allows every subdomain. IP
         addresses and wildcards are refused. A name that resolves to a private
-        address stays blocked. The first entry makes the swarm's list replace
+        address stays blocked. The entry adds to the names each hive's own
-        the names each hive's own configuration allows. The hive reads a change
+        configuration allows. The hive reads a change within about 30 seconds.
-        within about 30 seconds.
       </p>
       <form class="ui-agent-limits-form" onSubmit={(e) => void save(e)}>
         <TextField

hive-c0re/src/workers/squid_allow.rs

@@ -15,19 +15,19 @@
 //! Squid reads both when it parses its configuration, so the path unit in
 //! `nix/host-modules/hive-squid.nix` reloads it when either changes.
 //!
-//! The swarm's list replaces the names `services.hyperhive.squid.allowedDomains`
+//! The names `services.hyperhive.squid.allowedDomains` lists ([`ENV_SEED_FILE`])
-//! lists ([`ENV_SEED_FILE`]), which stay the list until the swarm has one. A
+//! are always allowed: the swarm-wide names written are those plus the swarm's,
-//! file rendered from the swarm's list starts with [`SWARM_MARKER`]. The queue
+//! with the per-agent rules on top. Until the swarm has a list the files hold
-//! not answering for the bucket does not show that the swarm withdrew its list,
+//! just those names. A file from the swarm's list starts with [`SWARM_MARKER`];
-//! so a file with the marker is left alone then.
+//! the queue not answering for the bucket is no withdrawal, so a file with the
+//! marker is left alone then.
 //!
 //! Every entry is checked again with
 //! [`swarm_wire::egress_allow::normalize_domain`], since the names end up in
 //! squid's configuration, and skipped when it fails. An entry that
 //! [`ENV_ALWAYS_ALLOWED`] or another entry of its list covers is dropped,
-//! because squid ignores the narrower of two overlapping names in one ACL.
+//! because squid ignores the narrower of two overlapping names in one ACL. A
-//!
+//! failed read leaves both files as they are.
-//! A failed read leaves both files as they are.
 
 use std::path::{Path, PathBuf};
 use std::time::Duration;
@@ -54,7 +54,7 @@ pub const ENV_ALLOWED_FILE: &str = "HIVE_SQUID_ALLOWED_FILE";
 pub const ENV_AGENT_ALLOW_FILE: &str = "HIVE_SQUID_AGENT_ALLOW_FILE";
 
 /// The file of names `services.hyperhive.squid.allowedDomains` lists, one per
-/// line, which the files hold until the swarm has a list.
+/// line, which every rendering of the swarm-wide names includes.
 pub const ENV_SEED_FILE: &str = "HIVE_SQUID_ALLOWED_SEED_FILE";
 
 /// The names, separated by spaces, that squid allows every authenticated user
@@ -191,7 +191,7 @@ fn sync_with(
 ) -> Result<Source> {
     let always: Vec<&str> = always.split_whitespace().collect();
     let rendered = match read {
-        Read::Current(list) => render(&list, agents, &always),
+        Read::Current(list) => render(&list, agents, &always, &seed_names(seed)?),
         Read::NotSet => {
             if super::squid_users::read_or_empty(files.allowed)?.starts_with(SWARM_MARKER) {
                 return Ok(Source::Kept);
@@ -251,18 +251,27 @@ struct Rendered {
     counts: Option<Counts>,
 }
 
-/// The files for the seed: its names as they are, and no per-agent rules.
+/// The names in the seed file, trimmed, without blank lines and comments. The
-fn seeded(seed: Option<&Path>) -> Result<Rendered> {
+/// file is this hive's own configuration, so its names are not re-checked.
-    let names = match seed {
+fn seed_names(seed: Option<&Path>) -> Result<Vec<String>> {
+    let text = match seed {
         Some(path) => super::squid_users::read_or_empty(path)?,
         None => String::new(),
     };
+    Ok(text
+        .lines()
+        .map(str::trim)
+        .filter(|line| !line.is_empty() && !line.starts_with('#'))
+        .map(str::to_owned)
+        .collect())
+}
+
+/// The files for the seed: its names as they are, and no per-agent rules.
+fn seeded(seed: Option<&Path>) -> Result<Rendered> {
     let mut allowed = SEED_MARKER.to_owned();
-    for line in names.lines().map(str::trim) {
+    for name in seed_names(seed)? {
-        if !line.is_empty() && !line.starts_with('#') {
+        allowed.push_str(&name);
-            allowed.push_str(line);
+        allowed.push('\n');
-            allowed.push('\n');
-        }
     }
     Ok(Rendered {
         allowed,
@@ -283,8 +292,9 @@ fn reduced(mut names: Vec<String>) -> Vec<String> {
     names
 }
 
-/// The files for `list`, for the agents this hive runs.
+/// The files for `list` and the seed's `own` names, for the agents this hive
-fn render(list: &AllowList, agents: &[String], always: &[&str]) -> Rendered {
+/// runs.
+fn render(list: &AllowList, agents: &[String], always: &[&str], own: &[String]) -> Rendered {
     let mut skipped = Vec::new();
     let mut swarm_wide = Vec::new();
     let mut per_agent: std::collections::BTreeMap<&str, Vec<String>> =
@@ -305,6 +315,7 @@ fn render(list: &AllowList, agents: &[String], always: &[&str]) -> Rendered {
             }
         }
     }
+    swarm_wide.extend(own.iter().map(|name| name.to_ascii_lowercase()));
     swarm_wide.retain(|d| !covered_by(d, always));
     let swarm_wide = reduced(swarm_wide);
 
@@ -366,7 +377,7 @@ mod tests {
     const ALWAYS: &[&str] = &["api.anthropic.com", ".claude.ai", "cache.nixos.org"];
 
     fn rendered(entries: &[(&str, Option<&str>)], mine: &[&str]) -> Rendered {
-        render(&list(entries), &agents(mine), ALWAYS)
+        render(&list(entries), &agents(mine), ALWAYS, &[])
     }
 
     #[test]
@@ -464,6 +475,7 @@ mod tests {
             &list(&[("api.anthropic.com", None), ("a.claude.ai", None)]),
             &agents(&["atlas"]),
             &[],
+            &[],
         );
         assert_eq!(
             bare.allowed,
@@ -545,18 +557,74 @@ mod tests {
         );
     }
 
+    /// The seed is the hive's own additive list: a swarm list, empty or not,
+    /// renders the seed's names with it.
     #[test]
-    fn a_swarm_list_replaces_the_seed_even_when_empty() {
+    fn a_swarm_list_adds_to_the_seed_even_when_empty() {
         let d = dir("forge.example\n");
         sync(&d, Read::NotSet).expect("seeded");
         assert!(text(&d.allowed()).contains("forge.example"), "control");
         let empty = Read::Current(AllowList::default());
         assert_eq!(sync(&d, empty).expect("renders"), Source::Swarm);
-        assert_eq!(text(&d.allowed()), SWARM_MARKER);
+        assert_eq!(text(&d.allowed()), "# source: swarm\nforge.example\n");
-        let named = Read::Current(list(&[("github.com", None), ("pypi.org", Some("atlas"))]));
+        let named = Read::Current(list(&[("github.com", None)]));
         sync(&d, named).expect("renders");
+        assert_eq!(
+            text(&d.allowed()),
+            "# source: swarm\nforge.example\ngithub.com\n"
+        );
+        assert_eq!(text(&d.agent_allow()), SWARM_MARKER);
+    }
+
+    #[test]
+    fn a_per_agent_entry_renders_on_top_of_the_seed() {
+        let d = dir("forge.example\n");
+        let read = Read::Current(list(&[("pypi.org", Some("atlas"))]));
+        sync(&d, read).expect("renders");
+        assert_eq!(text(&d.allowed()), "# source: swarm\nforge.example\n");
+        assert_eq!(
+            text(&d.agent_allow()),
+            "# source: swarm\n\
+             acl agent_allow_atlas proxy_auth atlas\n\
+             acl agent_allow_atlas_dst dstdomain pypi.org\n\
+             http_access allow agent_allow_atlas agent_allow_atlas_dst\n"
+        );
+    }
+
+    #[test]
+    fn a_seed_name_the_swarm_repeats_or_squid_allows_anyway_is_written_once() {
+        // `cache.nixos.org` is always allowed; `.example.com` covers the
+        // swarm's `api.example.com`.
+        let d = dir("Forge.Example\n.example.com\ncache.nixos.org\n");
+        let read = Read::Current(list(&[
+            ("forge.example", None),
+            ("api.example.com", None),
+            ("github.com", None),
+        ]));
+        sync(&d, read).expect("renders");
+        assert_eq!(
+            text(&d.allowed()),
+            "# source: swarm\n.example.com\nforge.example\ngithub.com\n"
+        );
+    }
+
+    #[test]
+    fn no_seed_file_is_no_seed_names() {
+        let d = dir("");
+        std::fs::remove_file(d.seed()).expect("remove");
+        let read = Read::Current(list(&[("github.com", None)]));
+        sync_with(
+            read,
+            &agents(&["atlas"]),
+            None,
+            "",
+            &Files {
+                allowed: &d.allowed(),
+                agent_allow: &d.agent_allow(),
+            },
+        )
+        .expect("renders");
         assert_eq!(text(&d.allowed()), "# source: swarm\ngithub.com\n");
-        assert!(text(&d.agent_allow()).contains("acl agent_allow_atlas proxy_auth atlas"));
     }
 
     #[test]

nix/host-modules/hive-squid.nix

@@ -5,8 +5,8 @@
 # hive-c0re renders. A credential's agent either has the swarm's
 # `unrestricted_internet` flag, and may then reach any destination the deny
 # rules leave, or reaches only the listed names: the base list, the allow-list
-# the swarm keeps (rendered by hive-c0re beside the users file, with this
+# the swarm keeps plus this host's `allowedDomains` (rendered by hive-c0re
-# host's `allowedDomains` as the list until the swarm has one) and the names
+# beside the users file) and the names
 # the swarm lists for that agent alone. An agent sends its credential
 # once it has fetched it (../agent-modules/egress-proxy.nix); the exporter,
 # journal forwarding and the self-check below show the proxy works.
@@ -32,8 +32,8 @@ let
   # The names squid allows every authenticated user whatever the swarm lists.
   # hive-c0re skips an allow-list entry that one of them already covers.
   alwaysAllowed = base ++ [ cfg.selfcheck.canary ];
-  # What `allowedFile` holds until the swarm has an allow-list: this hive's
+  # What `allowedFile` starts as and always includes: this hive's own
-  # own `allowedDomains`, one per line. The canary is in the configuration.
+  # `allowedDomains`, one per line. The canary is in the configuration.
   seedFile = pkgs.writeText "squid-allowed-seed" (
     lib.concatMapStrings (d: d + "\n") (
       lib.unique (lib.subtractLists [ cfg.selfcheck.canary ] cfg.allowedDomains)
@@ -254,12 +254,13 @@ in
         "crates.io"
       ];
       description = ''
-        Hostnames squid lets a client reach, by CONNECT or plain HTTP, until the
+        Hostnames squid always lets a client reach, by CONNECT or plain HTTP, on
-        swarm has an allow-list of its own: the list the swarm's controller
+        this hive. The entries the swarm's controller keeps
-        keeps (`/api/egress/allowed-domains`) replaces this one as soon as it
+        (`/api/egress/allowed-domains`) add to this list and never replace it:
-        holds an entry or has had one, and this option is the list again only
+        hive-c0re writes these names and the swarm-wide entries to
-        while the controller has none, for instance because it predates the
+        `allowedFile`. A hive-c0re that predates this rule replaced this list
-        feature. hive-c0re writes whichever list applies to `allowedFile`.
+        with the controller's once the controller held an entry, so every hive
+        must run the new one before the controller creates its first entry.
         Each entry matches that exact name, or that name and its subdomains
         with a leading dot. Everything else gets a 403, except
         for an unrestricted user. The swarm's own service names are not
@@ -340,9 +341,9 @@ in
       description = ''
         The names squid lets every restricted user reach, one per line, read as
         part of its allow ACL beside the self-check canary. It starts as a copy
-        of `allowedDomains`. hive-c0re rewrites it every pass: with the swarm's
+        of `allowedDomains`. hive-c0re rewrites it every pass: with
-        swarm-wide entries once the controller has a list, otherwise with
+        `allowedDomains` and the swarm-wide entries once the controller has a
-        `allowedDomains`. Squid is reloaded when it changes. Owned by
+        list, otherwise with `allowedDomains` alone. Squid is reloaded when it changes. Owned by
         `hive-core` and readable by the `squid` group.
       '';
     };
@@ -369,8 +370,8 @@ in
       readOnly = true;
       defaultText = lib.literalMD "`allowedDomains`, one per line, minus the self-check canary";
       description = ''
-        The names `allowedFile` starts as and hive-c0re writes to it again while
+        The names `allowedFile` starts as and hive-c0re writes into it on every
-        the swarm has no allow-list. Read-only.
+        pass, with or without a swarm allow-list. Read-only.
       '';
     };
 

nix/vm-tests/hive-squid.nix

@@ -520,13 +520,13 @@ pkgs.testers.runNixOSTest {
         wait_status("other.t.local", RUTH, "403")
         assert connect("other.t.local", UMA) == "200", "control: uma is still listed"
 
-    with subtest("the swarm's allow-list replaces the hive's names, opens one to every restricted user, and a per-agent rule opens one to that agent alone"):
+    with subtest("the swarm's allow-list adds to the hive's names, opens one to every restricted user, and a per-agent rule opens one to that agent alone"):
         seeded = proxy.succeed(f"cat {ALLOWED}")
         assert connect("swarm.t.local", RUTH) == "403", "control: not allowed yet"
         assert connect("agentonly.t.local", ATLAS) == "403", "control: not allowed yet"
         assert connect("forge.t.local", RUTH) == "200", "control: the hive's own name"
         write_allow(
-            "# source: swarm\nswarm.t.local\n",
+            "# source: swarm\n" + seeded + "swarm.t.local\n",
             "# source: swarm\n"
             "acl agent_allow_atlas proxy_auth atlas\n"
             "acl agent_allow_atlas_dst dstdomain agentonly.t.local priv10.t.local\n"
@@ -537,13 +537,13 @@ pkgs.testers.runNixOSTest {
         assert connect("swarm.t.local", ATLAS) == "200", "swarm-wide, another user"
         assert connect("agentonly.t.local", RUTH) == "403", "the per-agent name, another restricted user"
         assert connect("agentonly.t.local") == "407", "no credential"
-        assert connect("forge.t.local", RUTH) == "403", "the hive's own name is replaced"
+        assert connect("forge.t.local", RUTH) == "200", "the hive's own name stays"
         assert connect("priv10.t.local", ATLAS) == "403", "a per-agent name that resolves privately"
         assert connect("agentonly.t.local", UMA) == "200", "control: unrestricted reaches it anyway"
         write_allow(seeded, "")
         wait_status("swarm.t.local", RUTH, "403")
         wait_status("agentonly.t.local", ATLAS, "403")
-        assert connect("forge.t.local", RUTH) == "200", "the hive's own name is back"
+        assert connect("forge.t.local", RUTH) == "200", "the hive's own name after the swarm list is gone"
 
     with subtest("squid starts on a missing list, treats every user as restricted, and recovers when it appears"):
         proxy.succeed(f"rm {LIST}")

swarm-controller/README.md

@@ -66,8 +66,8 @@ The swarm's control plane. The swarm UI and `swarmctl` are its clients.
   hive's egress proxy, each for every agent or for one, as one value in the
   `egress-allow` KV bucket, which this daemon alone writes
   (`GET`/`POST`/`DELETE /api/egress/allowed-domains`); every hive reads it and
-  renders the proxy's allow-list files, and it replaces the list the hive's own
+  renders the proxy's allow-list files, which add to the names the hive's own
-  configuration names. It also keeps a `derived` entry per agent for the host
+  configuration lists. It also keeps a `derived` entry per agent for the host
   of its stored backend environment's `ANTHROPIC_BASE_URL` →
   [`docs/networking/network.md`](../docs/networking/network.md#egress-proxy-squid).
 - **Agent instructions** — each agent's `AGENTS.md` as its harness last

swarm-controller/src/backend_egress.rs

@@ -508,7 +508,7 @@ mod tests {
                 .await
                 .expect("the queue answers")
                 .is_none(),
-            "no bucket: each hive keeps its own list"
+            "no bucket: nothing was written"
         );
         // Control: a host does create it.
         put(

swarm-controller/src/egress_allow.rs

@@ -5,8 +5,9 @@
 //! (`swarm_wire::egress_allow`), which this daemon alone writes. Every hive
 //! reads it and renders the entries for the agents it runs: a swarm-wide entry
 //! joins the proxy's allow list for everyone, an entry with an agent is that
-//! agent's alone. Once the key exists it replaces the list a hive's own nix
+//! agent's alone. A hive always allows the names its own nix configuration
-//! configuration names, so an empty list is a list.
+//! lists (`services.hyperhive.squid.allowedDomains`) as well, so these entries
+//! only add to them.
 //!
 //! Each name goes into the proxy's configuration, so it is checked here with
 //! `swarm_wire::egress_allow::normalize_domain` (the hive checks again) and
@@ -127,7 +128,7 @@ async fn open_or_create(
     path = "/api/egress/allowed-domains",
     responses(
         (status = 200, description = "every entry, swarm-wide and per agent, sorted; empty \
-            when the swarm has not set a list, in which case each hive keeps its own",
+            when the swarm has not set a list",
             body = AllowedDomains),
         (status = 503, description = "no swarm queue is configured on this host, or it is \
             not connected (problem+json)", body = String),
@@ -154,10 +155,8 @@ pub(crate) async fn get_allowed_domains(
 
 /// Allow one destination, for every agent or for one.
 ///
-/// Adding an entry that is already there changes nothing. The first entry
+/// Adding an entry that is already there changes nothing. The entry adds to
-/// makes the swarm's list replace each hive's own, so a name the hive's nix
+/// the names each hive's nix configuration already allows.
-/// configuration allows and this list lacks stops being allowed once hives read
-/// it.
 #[utoipa::path(
     post,
     path = "/api/egress/allowed-domains",
@@ -221,8 +220,8 @@ pub(crate) async fn add_allowed_domain(
 
 /// Stop allowing one destination for the scope it was added with.
 ///
-/// Removing an entry that is not there changes nothing. Removing the last one
+/// Removing an entry that is not there changes nothing. A name a hive's nix
-/// leaves an empty list, which still replaces each hive's own.
+/// configuration lists stays allowed on that hive.
 #[utoipa::path(
     delete,
     path = "/api/egress/allowed-domains",
@@ -300,8 +299,7 @@ pub(crate) async fn current(client: &async_nats::Client) -> anyhow::Result<Optio
 /// is written or `change` leaves it as it is. Answers whether a write landed.
 ///
 /// A list that was never written is written only when `change` makes it
-/// non-empty: the key's existence is what makes the swarm's list replace each
+/// non-empty, so a change that adds nothing creates no bucket or key.
-/// hive's own, so a change that adds nothing must not create it.
 ///
 /// # Errors
 /// A queue failure, or `MAX_ATTEMPTS` conflicting writes.
@@ -541,7 +539,7 @@ mod tests {
         assert_eq!(
             published(&client).await.as_deref(),
             Some(r#"{"entries":[]}"#),
-            "the last removal leaves an empty list, which still replaces a hive's own"
+            "the last removal leaves an empty list"
         );
     }
 
@@ -578,7 +576,7 @@ mod tests {
     }
 
     /// A read before anything was added answers an empty list and creates no
-    /// bucket, so a hive keeps its own list.
+    /// bucket.
     #[tokio::test]
     async fn a_read_before_any_entry_answers_empty_and_creates_no_bucket() {
         let nats = TestNats::start();