Watch
0
0
Fork
You've already forked hyperhive
0

Compare commits

..
5 changed files with 40 additions and 124 deletions

View file

@ -289,24 +289,9 @@ host. It never decrypts TLS, so it holds no signing key. It caches nothing.
restricted. Agents must not start sending credentials until every hive runs
the `hive-c0re` that writes it, or a legacy agent loses general internet.
- **Base list.** `services.hyperhive.squid.baseAllowedDomains` is open to every
authenticated user, whatever its flag: what the agent harness needs off the
swarm. The default is the hosts Claude Code requires
(<https://code.claude.com/docs/en/network-config>): `api.anthropic.com` for
the model API and the usage probe (`hive-agent/src/claude_usage_watch.rs`),
`claude.ai` for the account login, and `platform.claude.com` for the OAuth
token exchange and refresh. An agent whose `ANTHROPIC_BASE_URL` or ACP
provider points elsewhere needs that host added to the option: the value
sits in the agent's own environment file, which the host doesn't read. The
list omits the plugin marketplace clone from `github.com`, since a failed add
isn't fatal. The rules below still refuse an entry that resolves to a
private address.
- **Allow-list.** `services.hyperhive.squid.allowedDomains` defaults to empty,
plus the self-check canary (`services.hyperhive.squid.selfcheck.canary`), so
the canary widens the restricted list by that one name. The swarm's own
service names aren't on it: they resolve to the bridge or a private address,
which the rules below refuse, and agents reach them directly. Each entry
matches that exact name. A user on the unrestricted list reaches any
- **Allow-list.** `services.hyperhive.squid.allowedDomains` defaults to the
swarm's own service names (`services.hyperhive.swarm.serviceDomains`). Each
entry matches that exact name. A user on the unrestricted list reaches any
other name too, on ports 80 and 443. Everyone else gets a 403 for it.
- **Unlisted names.** For a user who isn't on the unrestricted list, refused
after the destination checks, which look the name up on the host.
@ -322,8 +307,7 @@ host. It never decrypts TLS, so it holds no signing key. It caches nothing.
answer between a check and the connect gains nothing. A listed name must
therefore resolve to a public address on the host's resolver: a split-horizon
name that answers a LAN address gets a 403, and so does the self-check
canary, which is why it isn't a swarm service name. Only `CONNECT` to port
443 and plain HTTP to ports 80 and 443 pass.
canary. Only `CONNECT` to port 443 and plain HTTP to ports 80 and 443 pass.
- **Metrics.** `squid-exporter` serves `prometheus-squid-exporter` on
`127.0.0.1:9301`. The hive collector scrapes it as job `squid` through
`services.hyperhive.otel.scrapeTargets`. `squid_up` and
@ -334,8 +318,7 @@ host. It never decrypts TLS, so it holds no signing key. It caches nothing.
client's source address.
- **Self-check.** `squid-selfcheck.timer` runs `squid-selfcheck.service`
every minute from the bridge IP, logging in as `selfcheck.squid`: `CONNECT`
to `services.hyperhive.squid.selfcheck.canary` (default: `cache.nixos.org`,
a public name that's also on the allow-list)
to `services.hyperhive.squid.selfcheck.canary` (default: the forge's name)
must answer 200, to `deny-canary.invalid` 403, and to the canary without a
credential 407. It then logs in as `selfcheck-unrestricted.squid`, which has
the same secret and is always on the unrestricted list, and a `CONNECT` to
@ -350,7 +333,8 @@ credential=present|absent unrestricted_canary=… unrestricted_status=…` line
written the users file, the line reads `credential=absent` or
`allow_status=407` and `ok=false`, and nothing else breaks. The
host's resolver looks up the canary, so a host whose resolver maps
that name to a loopback, bridge or private address fails the check.
that name to a loopback or bridge address (a single-host swarm with
`gateway.localHostsEntry`) fails the check.
## Container isolation

View file

@ -323,7 +323,7 @@ that value. Read it with
`swarm_queue_client::agent_permissions::unrestricted`. The hive reads it for
the agent's squid policy: [the egress proxy](../networking/network.md#egress-proxy-squid)
lets an unrestricted agent's credential reach any destination, and holds the
others to the base list and the allow-list. An agent sends its credential once it has fetched it, from its next start.
others to the allow-list. An agent sends its credential once it has fetched it, from its next start.
**How a change applies** —
- The agent's harness reads its own key every 10 seconds and builds every

View file

@ -25,7 +25,6 @@ let
denyCanary = "deny-canary.invalid";
allowed = lib.unique (cfg.allowedDomains ++ [ cfg.selfcheck.canary ]);
base = lib.unique cfg.baseAllowedDomains;
# The squid user the self-check logs in as: `swarm_secret_client::proxy::SELFCHECK_USER`.
selfcheckUser = "selfcheck.squid";
@ -64,9 +63,9 @@ let
# The IP-literal
# denies come first because `dstdomain` on an IP literal does a reverse
# lookup. The credential check sits right before the first allow, so only a
# request that would otherwise be let through gets the 407 challenge. Every
# user is then let through to the base names, an unrestricted user to any other
# destination on a safe port, and every other user to the listed names. The `dst` denies cost an
# request that would otherwise be let through gets the 407 challenge. An
# unrestricted user is then let through to any other destination on a safe
# port, and every other user to the listed names. The `dst` denies cost an
# unlisted name a lookup on the host's resolver before it is refused.
squidConf = ''
http_port ${listen}
@ -106,7 +105,6 @@ let
# an agent controls could match an allowed name.
acl ip_v4_host url_regex ^([a-z]+://)?[0-9.]+(:[0-9]+)?(/|$)
acl ip_v6_host url_regex ^([a-z]+://)?\[[0-9a-fA-F:.]+\](:[0-9]+)?(/|$)
${lib.optionalString (base != [ ]) "acl base_dst dstdomain ${lib.concatStringsSep " " base}"}
${lib.optionalString (
allowed != [ ]
) "acl allowed_dst dstdomain ${lib.concatStringsSep " " allowed}"}
@ -125,7 +123,6 @@ let
http_access deny to_bridge
http_access deny to_private
http_access deny !authed
${lib.optionalString (base != [ ]) "http_access allow base_dst"}
http_access allow unrestricted
${lib.optionalString (allowed != [ ]) "http_access deny !allowed_dst"}
${lib.optionalString (allowed != [ ]) "http_access allow allowed_dst"}
@ -216,55 +213,23 @@ in
allowedDomains = lib.mkOption {
type = lib.types.listOf lib.types.str;
default = [ ];
default = hyperhiveCfg.swarm.serviceDomains;
defaultText = lib.literalExpression "config.services.hyperhive.swarm.serviceDomains";
example = [
"forge.example.com"
"github.com"
"crates.io"
];
description = ''
Hostnames squid lets a client reach, by CONNECT or plain HTTP. Each
entry matches that exact name. Everything else gets a 403, except
for an unrestricted user. The swarm's own service names are not
listed: they resolve to the bridge or a private address, which the
destination rules refuse, and agents reach them directly. The
self-check canary is allowed on top of this list, so it widens the
restricted allow list by that one name. A name must
resolve, on this host's resolver, to a public address: one in a
loopback, link-local, private, shared (100.64.0.0/10), multicast or
reserved range (or is `::`), or in the bridge subnet, gets a 403 for
for an unrestricted user. The self-check canary is allowed on top of
this list. A name must resolve, on this host's resolver, to a public
address: one in a loopback, link-local, private, shared (100.64.0.0/10),
multicast or reserved range (or is `::`), or in the bridge subnet, gets a 403 for
every user.
'';
};
baseAllowedDomains = lib.mkOption {
type = lib.types.listOf lib.types.str;
default = [
"api.anthropic.com"
"claude.ai"
"platform.claude.com"
];
example = [
"api.anthropic.com"
"claude.ai"
"platform.claude.com"
"openrouter.ai"
];
description = ''
Hostnames every authenticated user may reach, whether or not its agent
holds the `unrestricted_internet` flag: what the agent harness needs
off the swarm to work at all. The defaults are the hosts Claude Code
requires. `api.anthropic.com` serves the model API and the usage
probe, `claude.ai` the account login, and `platform.claude.com` the
OAuth token exchange and refresh. An agent whose `claude` or ACP provider
points elsewhere (`ANTHROPIC_BASE_URL`, an `opencode` provider) needs
that host listed here: the value lives in the agent's own backend
environment, which this host does not read. Swarm-internal names are
not listed, since agents reach them directly. The destination denies
still apply to every entry, and the list is separate from
`allowedDomains`.
'';
};
usersFile = lib.mkOption {
type = lib.types.path;
default = "/var/lib/hive-squid/users";
@ -320,15 +285,15 @@ in
selfcheck.canary = lib.mkOption {
type = lib.types.str;
default = "cache.nixos.org";
default = hyperhiveCfg.swarm.forge.domain;
defaultText = lib.literalExpression "config.services.hyperhive.swarm.forge.domain";
description = ''
Hostname the self-check CONNECTs to on port 443 and expects squid to
let through. It is added to the allowed names, so every restricted
agent can reach it too. It must resolve to a public address: squid
resolves it with this host's resolver, not the hive's, and a name
that answers a loopback, bridge or other private address (a swarm
service name on a hive, a split-horizon LAN) gets a 403 before
authentication, and the self-check reports `ok=false`.
let through. Squid resolves it with this host's resolver, not the
hive's: a host whose resolver answers a loopback, bridge or other
private address for it (a single-host swarm with the gateway's
`localHostsEntry`, or a split-horizon LAN) gets a 403, and the
self-check reports `ok=false`.
'';
};
};
@ -373,9 +338,6 @@ in
systemd.services.squid-unrestricted-reload = {
description = "Reload squid to read its unrestricted-users list";
# Several list writes in a few seconds must not trip the start limit: a
# tripped limit also fails the path unit, and later writes then reload nothing.
startLimitIntervalSec = 0;
serviceConfig = {
Type = "oneshot";
ExecStart = "${config.systemd.package}/bin/systemctl try-reload-or-restart squid.service";
@ -384,10 +346,10 @@ in
assertions = [
{
assertion = !(builtins.elem cfg.selfcheck.unrestrictedCanary (allowed ++ base));
assertion = !(builtins.elem cfg.selfcheck.unrestrictedCanary allowed);
message = ''
services.hyperhive.squid.selfcheck.unrestrictedCanary must be a name outside
allowedDomains and baseAllowedDomains, or the self-check cannot tell an unrestricted user from a
allowedDomains, or the self-check cannot tell an unrestricted user from a
restricted one.
'';
}

View file

@ -497,13 +497,11 @@ let
lines = lib.splitString "\n" hiveOtel.services.squid.configText;
at = l: lib.lists.findFirstIndex (x: x == l) null lines;
allow = at "http_access allow allowed_dst";
base = at "http_access allow base_dst";
unrestricted = at "http_access allow unrestricted";
authed = at "http_access deny !authed";
firstAllow = lib.min base (lib.min unrestricted allow);
firstAllow = if unrestricted < allow then unrestricted else allow;
in
allow != null
&& base != null
&& unrestricted != null
&& authed != null
&& at "http_access deny to_localhost" != null
@ -528,21 +526,6 @@ let
&& builtins.elem "cache deny all" lines
&& lib.last (lib.filter (l: l != "") lines) == "http_access deny all";
}
{
# Control: the allow ACL line is found and holds the canary, so the
# absence of the swarm's service names is not a lookup of nothing.
name = "squid's default allow-list is the self-check canary alone and its base list is the Claude Code hosts, with no swarm service name";
ok =
let
lines = lib.splitString "\n" hiveOtel.services.squid.configText;
acl = lib.findFirst (l: lib.hasPrefix "acl allowed_dst dstdomain " l) "" (map lib.trim lines);
baseAcl = lib.findFirst (l: lib.hasPrefix "acl base_dst dstdomain " l) "" (map lib.trim lines);
in
acl == "acl allowed_dst dstdomain ${hiveOtel.services.hyperhive.squid.selfcheck.canary}"
&& baseAcl == "acl base_dst dstdomain api.anthropic.com claude.ai platform.claude.com"
&& hiveOtel.services.hyperhive.squid.selfcheck.canary == "cache.nixos.org"
&& !(builtins.elem hiveOtel.services.hyperhive.swarm.forge.domain (lib.splitString " " acl));
}
{
# The ranges are what keeps an unrestricted user on the public internet.
# Control: the bridge deny and the ACL line are found, so a range missing
@ -584,8 +567,6 @@ let
in
at "http_access allow unrestricted" != null
&& at "http_access deny !allowed_dst" != null
&& at "http_access deny !authed" < at "http_access allow base_dst"
&& at "http_access allow base_dst" < at "http_access allow unrestricted"
&& at "http_access allow unrestricted" < at "http_access deny !allowed_dst"
&& at "http_access deny !allowed_dst" < at "http_access allow allowed_dst"
&& builtins.elem ''acl unrestricted proxy_auth "${list}"'' lines
@ -603,7 +584,6 @@ let
svc = hiveOtel.systemd.services.squid-unrestricted-reload.serviceConfig;
in
path.pathConfig.PathChanged == list
&& hiveOtel.systemd.services.squid-unrestricted-reload.startLimitIntervalSec == 0
&& builtins.elem "paths.target" path.wantedBy
&& lib.hasSuffix "systemctl try-reload-or-restart squid.service" svc.ExecStart
&& builtins.elem "f ${list} 0640 hive-core squid - -" rules

View file

@ -20,6 +20,14 @@ let
type = lib.types.bool;
default = true;
};
swarm.serviceDomains = lib.mkOption {
type = lib.types.listOf lib.types.str;
default = [ "forge.t.local" ];
};
swarm.forge.domain = lib.mkOption {
type = lib.types.str;
default = "forge.t.local";
};
network = {
bridgeIp = lib.mkOption {
type = lib.types.str;
@ -149,16 +157,11 @@ pkgs.testers.runNixOSTest {
# has no resolver to answer for an unknown one before the client gives up.
"203.0.113.2" = [
"forge.t.local"
"cache.t.local"
"api.t.local"
"other.t.local"
"deny-canary.invalid"
];
"127.0.0.1" = [ "loop.t.local" ];
${bridgeIp} = [
"bridge.t.local"
"basebridge.t.local"
];
${bridgeIp} = [ "bridge.t.local" ];
"169.254.169.254" = [ "meta.t.local" ];
"fd00:99::2" = [ "ula.t.local" ];
"::" = [ "any6.t.local" ];
@ -174,11 +177,6 @@ pkgs.testers.runNixOSTest {
# Renders the users file the way hive-c0re does.
environment.systemPackages = [ pkgs.mkpasswd ];
services.hyperhive.squid.selfcheck.unrestrictedCanary = "other.t.local";
services.hyperhive.squid.selfcheck.canary = "cache.t.local";
services.hyperhive.squid.baseAllowedDomains = [
"api.t.local"
"basebridge.t.local"
];
services.hyperhive.squid.allowedDomains = [
"forge.t.local"
"loop.t.local"
@ -325,20 +323,12 @@ pkgs.testers.runNixOSTest {
with subtest("an allowed name is tunnelled with a credential and challenged without one"):
assert connect("forge.t.local", ATLAS) == "200", "allow canary"
assert connect("cache.t.local", ATLAS) == "200", "the self-check canary, allowed on top of the list"
assert connect("forge.t.local") == "407", "no credential"
assert connect("forge.t.local", "atlas:" + secrets.token_urlsafe(24)) == "407", "wrong password"
assert connect("forge.t.local", "nobody:" + PW["atlas"]) == "407", "unknown user"
# DES would accept the first eight characters; the hash must not.
assert connect("forge.t.local", "atlas:" + PW["atlas"][:8]) == "407", "a prefix of the password"
with subtest("a base-list name is open to a restricted user, with a credential, and nothing else is"):
assert connect("api.t.local", ATLAS) == "200", "restricted user, base name"
assert connect("api.t.local") == "407", "no credential"
assert connect("api.t.local", "atlas:" + secrets.token_urlsafe(24)) == "407", "wrong password"
assert connect("other.t.local", ATLAS) == "403", "control: an unlisted name is still refused"
assert connect("basebridge.t.local", ATLAS) == "403", "a base name that resolves to the bridge"
with subtest("with an empty list every user is restricted"):
assert connect("other.t.local", UMA) == "403", "uma, not listed yet"
assert connect("other.t.local", ATLAS) == "403"
@ -419,7 +409,7 @@ pkgs.testers.runNixOSTest {
proxy.succeed("systemctl start squid-selfcheck.service")
proxy.succeed(
"journalctl -u squid-selfcheck.service -o cat | "
"grep -F 'squid-selfcheck ok=true allow_canary=cache.t.local "
"grep -F 'squid-selfcheck ok=true allow_canary=forge.t.local "
"allow_status=200 deny_canary=deny-canary.invalid deny_status=403 "
"anon_status=407 credential=present "
"unrestricted_canary=other.t.local unrestricted_status=200'"
@ -431,7 +421,7 @@ pkgs.testers.runNixOSTest {
proxy.fail("systemctl start squid-selfcheck.service")
proxy.succeed(
"journalctl -u squid-selfcheck.service -o cat | "
"grep -F 'squid-selfcheck ok=false allow_canary=cache.t.local "
"grep -F 'squid-selfcheck ok=false allow_canary=forge.t.local "
"allow_status=200 deny_canary=deny-canary.invalid deny_status=403 "
"anon_status=407 credential=present "
"unrestricted_canary=other.t.local unrestricted_status=403'"
@ -444,7 +434,7 @@ pkgs.testers.runNixOSTest {
proxy.fail("systemctl start squid-selfcheck.service")
proxy.succeed(
"journalctl -u squid-selfcheck.service -o cat | "
"grep -F 'squid-selfcheck ok=false allow_canary=cache.t.local "
"grep -F 'squid-selfcheck ok=false allow_canary=forge.t.local "
"allow_status=407 deny_canary=deny-canary.invalid deny_status=407 "
"anon_status=407 credential=absent "
"unrestricted_canary=other.t.local unrestricted_status=407'"