egress: forward = reject for intercepted agents, direct_mac exempt in log mode too
This commit is contained in:
atlas
parent
dff662c6b3
commit
dad15ca855
1 changed files with 248 additions and 70 deletions
|
|
@ -234,8 +234,9 @@ when it speaks HTTP on port 80 or TLS on port 443 and its agent is `restricted`
|
|||
or in `log` mode: the host redirects that traffic into squid's intercept ports
|
||||
(see _Intercepted web traffic_ below). Everything else goes straight out through
|
||||
the NAT, as before: the `unrestricted` agents, the service containers and every
|
||||
other port and protocol. Nothing blocks that yet. The host logs those direct
|
||||
connections (see _Agent bridge egress log_ below).
|
||||
other port and protocol. By default nothing blocks that. The host logs the direct
|
||||
connections of the `restricted` and `log` agents, and `egress.forward =
|
||||
"reject"` refuses them (see _Agent bridge egress log_ below).
|
||||
|
||||
Squid filters on the destination hostname: a `CONNECT` target or a plain-HTTP
|
||||
host. It never decrypts TLS. The one TLS key it loads is a throwaway that the
|
||||
|
|
@ -651,15 +652,15 @@ Each agent container runs in a private network namespace with a dedicated
|
|||
veth pair attached to the bridge. The following table summarises what
|
||||
the nix side sets up unconditionally:
|
||||
|
||||
| effect | mechanism |
|
||||
| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| IP forwarding | `boot.kernel.sysctl."net.ipv4.ip_forward" = 1` |
|
||||
| Internet NAT | `networking.nat { enable = true; internalInterfaces = [ bridgeName ]; }` — MASQUERADE on packets leaving via any external NIC |
|
||||
| Loopback DROP | `nix/host-modules/hive-egress.nix` — own nftables table `inet hive_egress` (whichever firewall the host runs): drops bridge → `127.0.0.0/8` and `::1` traffic; the kernel default `route_localnet=0` is the first layer |
|
||||
| Forward log | `nix/host-modules/hive-egress.nix` — the same table's `forward` chain logs and counts what the host routes from the bridge to the uplink; it blocks nothing (see _Agent bridge egress log_ below) |
|
||||
| Source check | `nix/host-modules/hive-egress.nix` — the `bridge hive_egress` table drops a frame whose source MAC or address differs from the sending container's own (see _Agent bridge source check_ below) |
|
||||
| Gateway access | `networking.firewall.interfaces.<bridge>.allowedTCPPorts = [ 80 443 ]` — lets isolated agents (private netns, veth on bridge) reach nginx on the host |
|
||||
| c0re signal | `HIVE_NETWORK_BRIDGE`, `HIVE_NETWORK_SUBNET` in `systemd.services.hive-c0re.environment` — both **required**; `hive-c0re` refuses to start without them |
|
||||
| effect | mechanism |
|
||||
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| IP forwarding | `boot.kernel.sysctl."net.ipv4.ip_forward" = 1` |
|
||||
| Internet NAT | `networking.nat { enable = true; internalInterfaces = [ bridgeName ]; }` — MASQUERADE on packets leaving via any external NIC |
|
||||
| Loopback DROP | `nix/host-modules/hive-egress.nix` — own nftables table `inet hive_egress` (whichever firewall the host runs): drops bridge → `127.0.0.0/8` and `::1` traffic; the kernel default `route_localnet=0` is the first layer |
|
||||
| Forward log | `nix/host-modules/hive-egress.nix` — the same table's `forward` chain logs and counts what the host routes from the bridge to the uplink, and refuses it with `egress.forward = "reject"` (see _Agent bridge egress log_ below) |
|
||||
| Source check | `nix/host-modules/hive-egress.nix` — the `bridge hive_egress` table drops a frame whose source MAC or address differs from the sending container's own (see _Agent bridge source check_ below) |
|
||||
| Gateway access | `networking.firewall.interfaces.<bridge>.allowedTCPPorts = [ 80 443 ]` — lets isolated agents (private netns, veth on bridge) reach nginx on the host |
|
||||
| c0re signal | `HIVE_NETWORK_BRIDGE`, `HIVE_NETWORK_SUBNET` in `systemd.services.hive-c0re.environment` — both **required**; `hive-c0re` refuses to start without them |
|
||||
|
||||
`HIVE_NETWORK_SUBNET` is the host-side bridge IP + prefix (e.g.
|
||||
`10.42.0.1/24`), **not** the canonical network address. The Rust side
|
||||
|
|
@ -727,12 +728,42 @@ loopback — the hive-c0re admin socket is a UDS, not TCP.
|
|||
`services.hyperhive.network.egress.forward` sets what the `forward` chain of
|
||||
the `inet hive_egress` table does with traffic the host routes from the agent
|
||||
bridge to anywhere but the bridge and the `wg-hive` mesh interface. It takes
|
||||
`accept`, `log` (the default) or `drop`. The evaluation refuses `drop` for now:
|
||||
the traffic that squid doesn't handle (the unrestricted agents, the service
|
||||
containers, every port but 80 and 443) has no rule that lets it through yet. In
|
||||
`log` mode the chain blocks nothing. It logs each new flow
|
||||
and counts every packet.
|
||||
`accept`, `log` (the default) or `reject`. In `log` mode the chain blocks
|
||||
nothing. It logs each new flow and counts every packet. In `reject` mode it
|
||||
also refuses the flow.
|
||||
|
||||
- **Who is exempt.** In `log` and `reject` mode alike, the chain accepts the
|
||||
mesh traffic first, then the traffic of every MAC in the `direct_mac` set: the
|
||||
`unrestricted` agents and the service containers, which the `intercept` chain
|
||||
leaves alone too (see _Intercepted web traffic_). They leave no line, so a
|
||||
`hive-egress-would-drop` line is exactly a flow that `reject` would refuse,
|
||||
and a service container keeps every port and protocol under `reject`.
|
||||
- **What `reject` refuses.** Everything from the bridge to the uplink that the
|
||||
chain didn't exempt, whether a flow is new or not: TCP on any port but the
|
||||
80 and 443 that squid takes, UDP (DNS to an outside resolver, QUIC and NTP
|
||||
included), ICMP and IPv6. A TCP flow gets a reset, so the connect fails at
|
||||
once instead of stalling. Everything else gets ICMP administratively
|
||||
prohibited. The log lines of this mode carry the prefix `hive-egress-reject `
|
||||
(and `hive-egress-reject-unmetered `); the rate limit and the counter work as
|
||||
in `log` mode. Still open to every agent are the host's own addresses: the
|
||||
proxy ports, DNS and DHCP from the bridge address, and the gateway.
|
||||
- **Switching on.** Leave `log` running for about a week, then read the
|
||||
`hive-egress-would-drop` lines. Each is a flow that `reject` would refuse; a
|
||||
flow that belongs to a legitimate use needs a fix first (a service container
|
||||
missing from `direct_mac`, an agent that should be `unrestricted`). Then set
|
||||
`services.hyperhive.network.egress.forward = "reject"`. Making it the default
|
||||
is a one-line change of the option's default.
|
||||
- **What `reject` can break.**
|
||||
- A service container with no pin yet isn't in `direct_mac`, so the
|
||||
chain refuses it until `hive-egress-pin` binds its port. The bridge table
|
||||
drops its frames until then anyway.
|
||||
- A change of an agent's `unrestricted` flag reaches `direct_mac` within 30
|
||||
seconds at most, on the timer of `hive-egress-pin`. The chain refuses the
|
||||
agent in the gap.
|
||||
- An unreadable `unrestricted` file lists no agent, so `unrestricted` agents
|
||||
lose every non-web flow.
|
||||
- An agent in per-agent `log` mode keeps its web traffic, which squid only
|
||||
observes, and loses every other port and protocol.
|
||||
- **Which traffic.** IPv4 and IPv6 alike (the table is `inet`). The chain never
|
||||
sees traffic to the host's own addresses, because the host delivers it
|
||||
locally: a connection to squid on `<bridgeIp>:3128`, to dnsmasq or to the
|
||||
|
|
|
|||
Loading…
Reference in a new issue