Watch
0
0
Fork
You've already forked hyperhive
0

load the agent-bridge to loopback drop in its own nft table

This commit is contained in:
atlas 2026-11-11 15:45:56 +01:00 • committed by mara
commit eb39a90057
3 changed files with 186 additions and 19 deletions

View file

@ -21,6 +21,7 @@
./hive-matrix-service.nix
./hive-matrix.nix
./hive-network.nix
./hive-egress.nix
./hive-priv.nix
./hive-archive-backup.nix
./hive-priv-cache.nix

View file

@ -0,0 +1,86 @@
# The hive's own nftables table, `inet hive_egress`, loaded by a systemd
# oneshot. It is independent of the NixOS firewall: `networking.firewall`
# renders only the rules of the backend in use (`iptables` unless
# `networking.nftables.enable` is set), so a rule that has to hold on every
# host lives here, in a table no firewall backend writes. Only
# `networking.nftables.flushRuleset` removes it; the unit re-applies it then.
#
# The table holds one chain: traffic from the agent bridge to host
# loopback is dropped. The kernel does the same by default (a packet for
# 127.0.0.0/8 arriving on a non-loopback interface is a martian while
# `route_localnet` is 0); this rule is the second layer, and the assertion
# below keeps the first one on.
{
lib,
config,
pkgs,
...
}:
let
cfg = config.services.hyperhive.network;
nft = lib.getExe pkgs.nftables;
# One transaction: `add` + `delete` drop an old copy of the table and the
# definition recreates it, so the file loads the same on first start, on a
# reload and after something else flushed the ruleset.
ruleset = pkgs.writeText "hive-egress.nft" ''
add table inet hive_egress
delete table inet hive_egress
table inet hive_egress {
chain input {
type filter hook input priority filter - 10; policy accept;
iifname "${cfg.bridgeName}" ip daddr 127.0.0.0/8 counter drop
iifname "${cfg.bridgeName}" ip6 daddr ::1 counter drop
}
}
'';
routeLocalnetKeys = [
"net.ipv4.conf.all.route_localnet"
"net.ipv4.conf.default.route_localnet"
"net.ipv4.conf.${cfg.bridgeName}.route_localnet"
];
isOn =
v:
builtins.elem v [
1
"1"
true
];
in
{
config = lib.mkIf cfg.enable {
assertions = [
{
assertion = !(lib.any (k: isOn (config.boot.kernel.sysctl.${k} or 0)) routeLocalnetKeys);
message = ''
boot.kernel.sysctl must not turn on route_localnet (${lib.concatStringsSep ", " routeLocalnetKeys}).
With it on, the kernel routes a packet from the agent bridge to 127.0.0.0/8
instead of dropping it as a martian, which leaves the hive_egress
table as the only barrier between an agent and the host's loopback services.
'';
}
];
systemd.services.hive-egress = {
description = "hive nftables table (agent bridge to host loopback drop)";
wantedBy = [ "multi-user.target" ];
wants = [ "network-pre.target" ];
before = [ "network-pre.target" ];
# `nftables.service` runs `flush ruleset` where `networking.nftables.flushRuleset`
# is on: start after it, follow its restarts and re-apply on its reload.
after = [ "nftables.service" ];
partOf = [ "nftables.service" ];
unitConfig.ReloadPropagatedFrom = [ "nftables.service" ];
reloadIfChanged = true;
serviceConfig = {
Type = "oneshot";
RemainAfterExit = true;
ExecStart = "${nft} -f ${ruleset}";
ExecReload = "${nft} -f ${ruleset}";
ExecStop = "-${nft} delete table inet hive_egress";
};
};
};
}

View file

@ -122,7 +122,7 @@ in
**The host service must bind an address reachable from the bridge** —
`0.0.0.0` or the bridge IP (`bridgeIp`) — not loopback-only. The
bridge→`127.0.0.0/8` DROP rule (defence-in-depth) is unchanged: this
bridge→`127.0.0.0/8` DROP rule (`hive-egress.nix`) still applies: this
only opens the firewall, it does not bridge loopback. A service that
binds `127.0.0.1` only is still unreachable; rebind it to `0.0.0.0`.
@ -276,12 +276,6 @@ in
internalInterfaces = [ cfg.bridgeName ];
};
# Defence-in-depth: DROP bridge→loopback so compromised agents can't
# reach host-loopback services even via routing table leaks.
networking.firewall.extraInputRules = ''
ip saddr ${cfg.bridgeIp}/${toString cfg.bridgePrefixLength} ip daddr 127.0.0.0/8 drop
'';
# Allow isolated agents to reach the gateway (nginx on the host, shared
# netns). Port 80 covers `http://forge.<domain>`, per-agent UI proxies,
# and any other HTTP services the gateway fronts. Port 443 for HTTPS.
@ -304,7 +298,7 @@ in
# Host port exposure: open each `exposeHostPorts` entry on the bridge
# firewall so agents can reach a host service at `<bridgeIp>:P`. The host
# service must bind `0.0.0.0` or the bridge IP (a loopback-only bind stays
# unreachable — the bridge→127.0.0.0/8 DROP rule above is unchanged). This
# unreachable — the bridge→127.0.0.0/8 DROP rule in hive-egress.nix still applies). This
# is firewall-only by design: a host service that binds `0.0.0.0` already
# serves the bridge IP, so an extra bridge-IP proxy would only collide
# (EADDRINUSE) with it. Merges with the [ 80 443 ] gateway ports above.