Watch
0
0
Fork
You've already forked hyperhive
0

egress: document the allowed ssh connections

This commit is contained in:
atlas 2026-11-20 03:25:50 +01:00
commit eab2dcb0a2
5 changed files with 73 additions and 19 deletions

View file

@ -60,9 +60,11 @@ whole file, at that path. To work inside the agent's container instead, see
key changed and you didn't change the machine, don't clear it: something else
answers at that address.
This doesn't open the network. If you set the hive's `egress.forward` to
`reject`, the hive refuses the agent's direct connection to the machine's port
until a separate allowance exists; the default `log` lets it through. See
The SSH switch and the CA don't open the network. The default `egress.forward`
of `log` lets the agent's direct connection to the machine's port through. If
you set it to `reject`, the hive refuses the connection unless you gave the
agent a named connection to that host and port (see
[Named connections](#named-connections)). See
[the network page](../../4-implementation/networking/network.md).
## 2. Make a machine trust the CA
@ -111,8 +113,16 @@ host and the port. For example `k8s-lab-vm`, `root`, `muede-pc2`, `2222`.
The agent's ssh config then holds a `Host k8s-lab-vm` block with that host,
user and port, and the connection uses the same certificate as any other. The
block follows a change within ten seconds, and it's empty while the agent's SSH
switch is off, so one switch ends both the certificate and the names. A name
doesn't open the network: see the note under section 1. The routes and the
switch is off, so one switch ends both the certificate and the names.
A named connection also lets the agent through the hive's egress rule. Under
`egress.forward = "reject"` the hive accepts TCP from that agent to the host
and port of each of its names, and from no other agent. The hive looks the
host up itself, every ten minutes, and accepts every IPv4 address it answers
with, except one inside its own agent bridge subnet. A name must therefore
resolve on the hive host the way it does for you. If the lookup fails or
hangs, the hive keeps the last answer. A new name, a removed one or a
flipped switch reaches the hive's rule in about a minute. The routes and the
rules a connection must meet are on
[the SSH CA implementation page](../../4-implementation/swarm/ssh-ca.md#named-connections).

View file

@ -737,11 +737,21 @@ also refuses the flow.
leaves alone too (see _Intercepted web traffic_). They leave no line, so a
`hive-egress-would-drop` line is exactly a flow that `reject` would refuse,
and a service container keeps every port and protocol under `reject`.
- **Named SSH connections.** Right after the `direct_mac` rule the chain
accepts TCP from an agent's MAC to an address and port in the `agent_allow`
set, in both modes, so such a flow leaves no count and no log line either.
The set holds, for each agent whose `ssh` switch is on, the addresses its
connections' hosts resolve to and their ports (see
[the SSH CA page](../swarm/ssh-ca.md#egress)). `hive-c0re` renders them into
`services.hyperhive.deploy.hive-controller.egress.connectionsFile`
(`/var/lib/hive-squid/ssh-connections`), and `hive-egress-pin` fills the set
from the file like `direct_mac`, when the file changes and on its timer. The
set is empty while the file is missing or empty.
- **What `reject` refuses.** Everything from the bridge to the uplink that the
chain didn't exempt, whether a flow is new or not: TCP on any port but the
80 and 443 that squid takes, UDP (DNS to an outside resolver, QUIC and NTP
included), ICMP and IPv6. A TCP flow gets a reset, so the connect fails at
once instead of stalling. Everything else gets ICMP administratively
chain didn't exempt or allow, whether a flow is new or not: TCP on any port
but the 80 and 443 that squid takes, UDP (DNS to an outside resolver, QUIC
and NTP included), ICMP and IPv6. A TCP flow gets a reset, so the connect
fails at once instead of stalling. Everything else gets ICMP administratively
prohibited. The log lines of this mode carry the prefix `hive-egress-reject `
(and `hive-egress-reject-unmetered `); the rate limit and the counter work as
in `log` mode. Still open to every agent are the host's own addresses: the
@ -761,6 +771,11 @@ also refuses the flow.
agent in the gap.
- An unreadable `unrestricted` file lists no agent, so `unrestricted` agents
lose every non-web flow.
- A new named connection, or a flipped `ssh` switch, reaches the set about a
minute after the write: the hive reads the swarm every 30 seconds and the
file's change runs a pass at once. A host that moves to an address the hive
hasn't looked up yet gets refusals for up to ten minutes. An agent whose
connection names a host that never resolved has no allowance for it.
- An agent in per-agent `log` mode keeps its web traffic, which squid only
observes, and loses every other port and protocol.
- **Which traffic.** IPv4 and IPv6 alike (the table is `inet`). The chain never

View file

@ -147,7 +147,8 @@ stores a connection and by the agent before it renders one:
its own proxy, so no direct connection there can exist.
The swarm doesn't check the hive's own bridge subnet, because each hive sets its
own and an agent can move between hives.
own and an agent can move between hives. Each hive drops an address of its own
subnet when it resolves a host (see [Egress](#egress)).
`hive-agent`'s `ssh_connections.rs` renders the list each time the harness reads
the record (every ten seconds) into `/run/hive-agent-ssh-connections/config`:
@ -159,11 +160,35 @@ agent user, because ssh refuses an included file that another user owns.
A controller older than the field drops the list when it next writes the
record through `PUT /permissions`.
## Not covered
## Egress
Egress. The default `egress.forward = "log"` lets a direct connection from an
agent to a machine's SSH port through, and `reject` blocks it; there is no
per-agent allowance for it yet.
The hive's forward chain accepts a named connection of an agent that has `ssh`
on, whatever `egress.forward` is (`log` or `reject`). Each hive does this for
the agents it has:
1. `hive-c0re`'s `workers::egress_connections` reads the permissions the hive
last read from the swarm and, for each agent with `ssh` on, resolves the
`host` of each connection with the host's resolver. An IPv4 literal passes
through. It asks the hosts of a pass together, each for at most five seconds,
so a hung resolver delays the file by that long and no more. It keeps an
answer for ten minutes and keeps the last answer when a lookup fails or
times out (one warning, until the name resolves again). It drops loopback,
link-local, multicast and IPv6 answers, and any address inside the hive's
bridge subnet (`HIVE_NETWORK_SUBNET`) with one warning per host, because a
flow to the bridge never reaches the forward chain. It writes
`<agent> <ipv4> <port>` lines to `egress.connectionsFile`, beside squid's
users file.
2. `hive-priv egress-pin` maps each agent to the MAC of its port and fills the
`agent_allow` set of the `inet hive_egress` table, in the same transaction
as `direct_mac`. `hive-egress-connections.path` runs a pass when the file
changes.
3. The forward chain accepts `ether saddr . ip daddr . tcp dport @agent_allow`
right after the `direct_mac` rule, so a matching flow is neither counted
nor logged.
The allowance is for TCP and IPv4 only: the agent bridge has no IPv6
addressing. An agent whose port has no binding yet has no allowance until it
gets one. See [the network page](../networking/network.md).
## Revocation

View file

@ -179,7 +179,9 @@ let
# so it allows nothing those refuse; a `reject` ends the packet. The MACs
# in `direct_mac` (the unrestricted agents and the service
# containers, the traffic the intercept chain leaves alone) come first in
# both modes, so a `log` line names what `reject` would refuse. Traffic to
# both modes, so a `log` line names what `reject` would refuse. The TCP
# connections in `agent_allow` (an agent's `ssh_connections`, resolved by
# hive-c0re) are accepted next, with the same effect. Traffic to
# the host's own addresses (squid, dnsmasq, the gateway) is delivered
# locally and never reaches the forward hook.
forwardChain = ''

View file

@ -98,12 +98,14 @@ pub struct AgentPermissions {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub max_repos: Option<u32>,
/// Whether the agent may have its SSH key signed by the swarm's SSH CA. Absent
/// is off. Only the controller reads it: it attaches the SSH signing policy
/// to the agent's store login role while this is `true`.
/// is off. The controller attaches the SSH signing policy to the agent's store
/// login role while this is `true`, and a hive lets the agent's
/// `ssh_connections` through its forward chain while it is.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub ssh: Option<bool>,
/// The named SSH connections the operator declared for the agent. The
/// agent renders them into its ssh config while `ssh` is on. Each passes
/// agent renders them into its ssh config and its hive allows them on its
/// forward chain, while `ssh` is on. Each passes
/// [`crate::ssh_connection::SshConnection::checked`] before it is stored,
/// and names are unique. A controller built before this field decodes the
/// record without it and stores it back without it.