egress: document the allowed ssh connections
This commit is contained in:
parent
8ece0b7e0b
commit
eab2dcb0a2
5 changed files with 73 additions and 19 deletions
|
|
@ -60,9 +60,11 @@ whole file, at that path. To work inside the agent's container instead, see
|
|||
key changed and you didn't change the machine, don't clear it: something else
|
||||
answers at that address.
|
||||
|
||||
This doesn't open the network. If you set the hive's `egress.forward` to
|
||||
`reject`, the hive refuses the agent's direct connection to the machine's port
|
||||
until a separate allowance exists; the default `log` lets it through. See
|
||||
The SSH switch and the CA don't open the network. The default `egress.forward`
|
||||
of `log` lets the agent's direct connection to the machine's port through. If
|
||||
you set it to `reject`, the hive refuses the connection unless you gave the
|
||||
agent a named connection to that host and port (see
|
||||
[Named connections](#named-connections)). See
|
||||
[the network page](../../4-implementation/networking/network.md).
|
||||
|
||||
## 2. Make a machine trust the CA
|
||||
|
|
@ -111,8 +113,16 @@ host and the port. For example `k8s-lab-vm`, `root`, `muede-pc2`, `2222`.
|
|||
The agent's ssh config then holds a `Host k8s-lab-vm` block with that host,
|
||||
user and port, and the connection uses the same certificate as any other. The
|
||||
block follows a change within ten seconds, and it's empty while the agent's SSH
|
||||
switch is off, so one switch ends both the certificate and the names. A name
|
||||
doesn't open the network: see the note under section 1. The routes and the
|
||||
switch is off, so one switch ends both the certificate and the names.
|
||||
|
||||
A named connection also lets the agent through the hive's egress rule. Under
|
||||
`egress.forward = "reject"` the hive accepts TCP from that agent to the host
|
||||
and port of each of its names, and from no other agent. The hive looks the
|
||||
host up itself, every ten minutes, and accepts every IPv4 address it answers
|
||||
with, except one inside its own agent bridge subnet. A name must therefore
|
||||
resolve on the hive host the way it does for you. If the lookup fails or
|
||||
hangs, the hive keeps the last answer. A new name, a removed one or a
|
||||
flipped switch reaches the hive's rule in about a minute. The routes and the
|
||||
rules a connection must meet are on
|
||||
[the SSH CA implementation page](../../4-implementation/swarm/ssh-ca.md#named-connections).
|
||||
|
||||
|
|
|
|||
|
|
@ -737,11 +737,21 @@ also refuses the flow.
|
|||
leaves alone too (see _Intercepted web traffic_). They leave no line, so a
|
||||
`hive-egress-would-drop` line is exactly a flow that `reject` would refuse,
|
||||
and a service container keeps every port and protocol under `reject`.
|
||||
- **Named SSH connections.** Right after the `direct_mac` rule the chain
|
||||
accepts TCP from an agent's MAC to an address and port in the `agent_allow`
|
||||
set, in both modes, so such a flow leaves no count and no log line either.
|
||||
The set holds, for each agent whose `ssh` switch is on, the addresses its
|
||||
connections' hosts resolve to and their ports (see
|
||||
[the SSH CA page](../swarm/ssh-ca.md#egress)). `hive-c0re` renders them into
|
||||
`services.hyperhive.deploy.hive-controller.egress.connectionsFile`
|
||||
(`/var/lib/hive-squid/ssh-connections`), and `hive-egress-pin` fills the set
|
||||
from the file like `direct_mac`, when the file changes and on its timer. The
|
||||
set is empty while the file is missing or empty.
|
||||
- **What `reject` refuses.** Everything from the bridge to the uplink that the
|
||||
chain didn't exempt, whether a flow is new or not: TCP on any port but the
|
||||
80 and 443 that squid takes, UDP (DNS to an outside resolver, QUIC and NTP
|
||||
included), ICMP and IPv6. A TCP flow gets a reset, so the connect fails at
|
||||
once instead of stalling. Everything else gets ICMP administratively
|
||||
chain didn't exempt or allow, whether a flow is new or not: TCP on any port
|
||||
but the 80 and 443 that squid takes, UDP (DNS to an outside resolver, QUIC
|
||||
and NTP included), ICMP and IPv6. A TCP flow gets a reset, so the connect
|
||||
fails at once instead of stalling. Everything else gets ICMP administratively
|
||||
prohibited. The log lines of this mode carry the prefix `hive-egress-reject `
|
||||
(and `hive-egress-reject-unmetered `); the rate limit and the counter work as
|
||||
in `log` mode. Still open to every agent are the host's own addresses: the
|
||||
|
|
@ -761,6 +771,11 @@ also refuses the flow.
|
|||
agent in the gap.
|
||||
- An unreadable `unrestricted` file lists no agent, so `unrestricted` agents
|
||||
lose every non-web flow.
|
||||
- A new named connection, or a flipped `ssh` switch, reaches the set about a
|
||||
minute after the write: the hive reads the swarm every 30 seconds and the
|
||||
file's change runs a pass at once. A host that moves to an address the hive
|
||||
hasn't looked up yet gets refusals for up to ten minutes. An agent whose
|
||||
connection names a host that never resolved has no allowance for it.
|
||||
- An agent in per-agent `log` mode keeps its web traffic, which squid only
|
||||
observes, and loses every other port and protocol.
|
||||
- **Which traffic.** IPv4 and IPv6 alike (the table is `inet`). The chain never
|
||||
|
|
|
|||
|
|
@ -147,7 +147,8 @@ stores a connection and by the agent before it renders one:
|
|||
its own proxy, so no direct connection there can exist.
|
||||
|
||||
The swarm doesn't check the hive's own bridge subnet, because each hive sets its
|
||||
own and an agent can move between hives.
|
||||
own and an agent can move between hives. Each hive drops an address of its own
|
||||
subnet when it resolves a host (see [Egress](#egress)).
|
||||
|
||||
`hive-agent`'s `ssh_connections.rs` renders the list each time the harness reads
|
||||
the record (every ten seconds) into `/run/hive-agent-ssh-connections/config`:
|
||||
|
|
@ -159,11 +160,35 @@ agent user, because ssh refuses an included file that another user owns.
|
|||
A controller older than the field drops the list when it next writes the
|
||||
record through `PUT /permissions`.
|
||||
|
||||
## Not covered
|
||||
## Egress
|
||||
|
||||
Egress. The default `egress.forward = "log"` lets a direct connection from an
|
||||
agent to a machine's SSH port through, and `reject` blocks it; there is no
|
||||
per-agent allowance for it yet.
|
||||
The hive's forward chain accepts a named connection of an agent that has `ssh`
|
||||
on, whatever `egress.forward` is (`log` or `reject`). Each hive does this for
|
||||
the agents it has:
|
||||
|
||||
1. `hive-c0re`'s `workers::egress_connections` reads the permissions the hive
|
||||
last read from the swarm and, for each agent with `ssh` on, resolves the
|
||||
`host` of each connection with the host's resolver. An IPv4 literal passes
|
||||
through. It asks the hosts of a pass together, each for at most five seconds,
|
||||
so a hung resolver delays the file by that long and no more. It keeps an
|
||||
answer for ten minutes and keeps the last answer when a lookup fails or
|
||||
times out (one warning, until the name resolves again). It drops loopback,
|
||||
link-local, multicast and IPv6 answers, and any address inside the hive's
|
||||
bridge subnet (`HIVE_NETWORK_SUBNET`) with one warning per host, because a
|
||||
flow to the bridge never reaches the forward chain. It writes
|
||||
`<agent> <ipv4> <port>` lines to `egress.connectionsFile`, beside squid's
|
||||
users file.
|
||||
2. `hive-priv egress-pin` maps each agent to the MAC of its port and fills the
|
||||
`agent_allow` set of the `inet hive_egress` table, in the same transaction
|
||||
as `direct_mac`. `hive-egress-connections.path` runs a pass when the file
|
||||
changes.
|
||||
3. The forward chain accepts `ether saddr . ip daddr . tcp dport @agent_allow`
|
||||
right after the `direct_mac` rule, so a matching flow is neither counted
|
||||
nor logged.
|
||||
|
||||
The allowance is for TCP and IPv4 only: the agent bridge has no IPv6
|
||||
addressing. An agent whose port has no binding yet has no allowance until it
|
||||
gets one. See [the network page](../networking/network.md).
|
||||
|
||||
## Revocation
|
||||
|
||||
|
|
|
|||
|
|
@ -179,7 +179,9 @@ let
|
|||
# so it allows nothing those refuse; a `reject` ends the packet. The MACs
|
||||
# in `direct_mac` (the unrestricted agents and the service
|
||||
# containers, the traffic the intercept chain leaves alone) come first in
|
||||
# both modes, so a `log` line names what `reject` would refuse. Traffic to
|
||||
# both modes, so a `log` line names what `reject` would refuse. The TCP
|
||||
# connections in `agent_allow` (an agent's `ssh_connections`, resolved by
|
||||
# hive-c0re) are accepted next, with the same effect. Traffic to
|
||||
# the host's own addresses (squid, dnsmasq, the gateway) is delivered
|
||||
# locally and never reaches the forward hook.
|
||||
forwardChain = ''
|
||||
|
|
|
|||
|
|
@ -98,12 +98,14 @@ pub struct AgentPermissions {
|
|||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub max_repos: Option<u32>,
|
||||
/// Whether the agent may have its SSH key signed by the swarm's SSH CA. Absent
|
||||
/// is off. Only the controller reads it: it attaches the SSH signing policy
|
||||
/// to the agent's store login role while this is `true`.
|
||||
/// is off. The controller attaches the SSH signing policy to the agent's store
|
||||
/// login role while this is `true`, and a hive lets the agent's
|
||||
/// `ssh_connections` through its forward chain while it is.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub ssh: Option<bool>,
|
||||
/// The named SSH connections the operator declared for the agent. The
|
||||
/// agent renders them into its ssh config while `ssh` is on. Each passes
|
||||
/// agent renders them into its ssh config and its hive allows them on its
|
||||
/// forward chain, while `ssh` is on. Each passes
|
||||
/// [`crate::ssh_connection::SshConnection::checked`] before it is stored,
|
||||
/// and names are unique. A controller built before this field decodes the
|
||||
/// record without it and stores it back without it.
|
||||
|
|
|
|||
Loading…
Reference in a new issue