vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

Compare commits

base: vinzenz:b66a09f65bd46a4bc1cd098dd4c7caafa03a420d
Branches Tags
vinzenz:main
..
compare: vinzenz:db4bfd4007553998305a40dbde5fc05e01d85166
Branches Tags
vinzenz:main
6 changed files with 21 additions and 71 deletions
Download patch file Download diff file Expand all files Collapse all files

19
docs/networking/network.md
View file

@ -116,13 +116,12 @@ schemes pick their own.
dnsmasq is **authoritative** for the hive's own zone (`<hive-domain>`) dnsmasq is **authoritative** for the hive's own zone (`<hive-domain>`)
plus whatever swarm-service names this host contributes via plus whatever swarm-service names this host contributes via
`gateway.localNames` — `forge.<swarm-domain>` and `chat.<swarm-domain>` `gateway.localNames` — `forge.<swarm-domain>` and `chat.<swarm-domain>`
(matrix) when this host runs those services, `auth.<swarm-domain>` (matrix) when this host runs those services, and `auth.<swarm-domain>`
when it runs authelia, and `hive-collector.<swarm-domain>` on every hive when it runs authelia — answering each with the bridge IP (where nginx
with `otel.enable` — answering each with the bridge IP (where nginx is reachable). Note forge and matrix are swarm-domain names, not
is reachable, and where the hive's own collector listens). Note forge and sub-domains of `<hive-domain>`: a swarm runs one forge and one
matrix are swarm-domain names, not sub-domains of `<hive-domain>`: a swarm homeserver, so their names belong to the swarm rather than to whichever
runs one forge and one homeserver, so their names belong to the swarm rather hive happens to host them. dnsmasq forwards everything else to the host's
than to whichever hive happens to host them. dnsmasq forwards everything else to the host's
own resolvers: it runs on the host and reads the host's own resolvers: it runs on the host and reads the host's
`/etc/resolv.conf` directly. Containers don't need to know the `/etc/resolv.conf` directly. Containers don't need to know the
upstream — they query the bridge IP and dnsmasq does the right thing upstream — they query the bridge IP and dnsmasq does the right thing
@ -188,10 +187,8 @@ that `otel.endpoint` is the _upstream_, which no agent ever dials. See
`services.hyperhive.network.exposeHostPorts = [ 4318 ];` opens each `services.hyperhive.network.exposeHostPorts = [ 4318 ];` opens each
listed TCP port `P` on the bridge-interface `allowedTCPPorts`, so an listed TCP port `P` on the bridge-interface `allowedTCPPorts`, so an
agent can connect to `<bridgeIp>:P` (for a dev OTLP collector, point its agent can connect to `<bridgeIp>:P` (point the collector endpoint at
endpoint at `http://<bridgeIp>:4318`, default `http://10.42.0.1:4318`). `http://<bridgeIp>:4318`, default `http://10.42.0.1:4318`).
The hive's own collector needs none of this: agents dial it by
`otel.collector.host`.
<!-- vale write-good.Passive = NO --> <!-- vale write-good.Passive = NO -->

9
docs/scheduler/observability.md
View file

@ -41,12 +41,9 @@ destination in its own right. With both, telemetry goes to both. See
**Telemetry leaves a hive exactly one way: through the collector that **Telemetry leaves a hive exactly one way: through the collector that
`enable` starts on the host.** Agents never talk to `endpoint` themselves — `enable` starts on the host.** Agents never talk to `endpoint` themselves —
they export unauthenticated to a collector address only their own containers they export unauthenticated to a bridge address only their own containers can
can reach. They dial it as reach (`http://<bridgeIp>:<collector port>`; the otel module opens that port
`http://hive-collector.<swarm domain>:<collector port>`, a name identical on on the bridge itself). That collector forwards to the swarm's
every hive, which each hive's own dnsmasq answers with its bridge IP, so an
agent's build carries no hive-specific address. The otel module opens that
port on the bridge itself. That collector forwards to the swarm's
([`swarm/services.md`](../swarm/services.md#telemetry-collector-otel)), the ([`swarm/services.md`](../swarm/services.md#telemetry-collector-otel)), the
single process holding the upstream credential and the only writer to the single process holding the upstream credential and the only writer to the
swarm's stores. No agent holds a copy, and neither does the hive. swarm's stores. No agent holds a copy, and neither does the hive.

3
hive-c0re/src/stats/otel_metrics.rs
View file

@ -2,8 +2,7 @@
//! sampled in [`super::container_stats`] feeds: it rides those gauges out to the //! sampled in [`super::container_stats`] feeds: it rides those gauges out to the
//! hive's own collector — the same first hop the agents use, named by the //! hive's own collector — the same first hop the agents use, named by the
//! standard `OTEL_EXPORTER_OTLP_ENDPOINT` (see `nix/host-modules/hive-c0re`, //! standard `OTEL_EXPORTER_OTLP_ENDPOINT` (see `nix/host-modules/hive-c0re`,
//! which sets it to the collector's bridge address where agents get its //! which sets it from the same binding it hands agents). No toggle of
//! swarm-wide name). No toggle of
//! its own, and no credential: a hive's collector takes unauthenticated OTLP //! its own, and no credential: a hive's collector takes unauthenticated OTLP
//! on the bridge, and the only hop that presents anything is the swarm tier's, //! on the bridge, and the only hop that presents anything is the swarm tier's,
//! which is the one that leaves the swarm. //! which is the one that leaves the swarm.

16
nix/host-modules/hive-c0re/environment.nix
View file

@ -83,13 +83,11 @@ in
# don't render no-op env lines. # don't render no-op env lines.
let let
otel = config.services.hyperhive.otel; otel = config.services.hyperhive.otel;
# The first hop, this hive's own collector, under the two spellings its # The first hop, bound once and consumed twice below: what agents are
# two kinds of caller need. Agents get the swarm-wide name, so their # handed, and where hive-c0re's own exporter sends. One binding so the
# builds carry nothing hive-specific; this daemon gets the bridge # address a hive tells its agents about and the one it uses itself
# address, because on the host the name may resolve to loopback # cannot drift apart.
# (`gateway.localHostsEntry`), where the receiver does not listen. firstHop = "http://${config.services.hyperhive.network.bridgeIp}:${toString otel.collector.port}";
agentFirstHop = "http://${otel.collector.host}:${toString otel.collector.port}";
hostFirstHop = "http://${config.services.hyperhive.network.bridgeIp}:${toString otel.collector.port}";
in in
{ {
# `otel.endpoint` means "where telemetry ultimately goes" and keeps # `otel.endpoint` means "where telemetry ultimately goes" and keeps
@ -97,7 +95,7 @@ in
# always this hive's own collector. Deriving it rather than # always this hive's own collector. Deriving it rather than
# redefining `endpoint` is what lets every existing deployment keep # redefining `endpoint` is what lets every existing deployment keep
# its configured value untouched. # its configured value untouched.
HYPERHIVE_OTEL_ENDPOINT = agentFirstHop; HYPERHIVE_OTEL_ENDPOINT = firstHop;
# hive-c0re's OWN container-resource exporter (stats/otel_metrics.rs) # hive-c0re's OWN container-resource exporter (stats/otel_metrics.rs)
# reads the STANDARD OTLP variable — the same one hive-metric and every # reads the STANDARD OTLP variable — the same one hive-metric and every
# agent read — and lets the SDK resolve the URL, which appends the # agent read — and lets the SDK resolve the URL, which appends the
@ -110,7 +108,7 @@ in
# replace this one: # replace this one:
# it is the agent-config transport meta.rs reads, and the SDK does not # it is the agent-config transport meta.rs reads, and the SDK does not
# know that name. # know that name.
OTEL_EXPORTER_OTLP_ENDPOINT = hostFirstHop; OTEL_EXPORTER_OTLP_ENDPOINT = firstHop;
# The first hop is the collector's OTLP/HTTP receiver, which speaks # The first hop is the collector's OTLP/HTTP receiver, which speaks
# protobuf regardless of what the upstream wants — `otel.protocol` # protobuf regardless of what the upstream wants — `otel.protocol`
# describes the *upstream* link, and the collector's own exporter is # describes the *upstream* link, and the collector's own exporter is

31
nix/host-modules/otel.nix
View file

@ -67,10 +67,8 @@ in
Enabling this also runs this hive's collector on the host: there is Enabling this also runs this hive's collector on the host: there is
exactly one way telemetry leaves this hive, and it is through that exactly one way telemetry leaves this hive, and it is through that
collector. Agents export unauthenticated to collector. Agents export unauthenticated to a bridge address only
{option}`services.hyperhive.otel.collector.host`, a name this hive's their own containers can reach.
resolver answers with a bridge address only its own containers can
reach.
That collector forwards to the swarm's That collector forwards to the swarm's
({option}`services.hyperhive.deploy.swarm-otel.enable`), which holds the ({option}`services.hyperhive.deploy.swarm-otel.enable`), which holds the
@ -205,28 +203,6 @@ in
''; '';
}; };
collector.host = lib.mkOption {
type = lib.types.str;
readOnly = true;
# Total on a null swarm domain, as in ./swarm-otel-service.nix: the
# required-domain assertion should be what an operator sees.
default = "hive-collector.${
if config.services.hyperhive.swarm.domain == null then
"invalid"
else
config.services.hyperhive.swarm.domain
}";
defaultText = lib.literalExpression ''"hive-collector.''${services.hyperhive.swarm.domain}"'';
description = ''
The name agents reach this hive's collector on. The same string on
every hive: each hive's own resolver answers it with its own bridge
IP, so an agent's build carries no hive-specific address.
No hive can claim this name as its domain:
`nix/reserved-hive-fragments.nix` forbids `hive` in any hive name.
'';
};
telemetryPort = lib.mkOption { telemetryPort = lib.mkOption {
type = lib.types.port; type = lib.types.port;
default = 8888; default = 8888;
@ -381,9 +357,6 @@ in
# the port on the bridge interface only. # the port on the bridge interface only.
services.hyperhive.network.exposeHostPorts = [ otel.collector.port ]; services.hyperhive.network.exposeHostPorts = [ otel.collector.port ];
# The name agents dial, answered with the bridge IP the receiver binds.
services.hyperhive.gateway.localNames = [ otel.collector.host ];
# …and the receiver binds that bridge address, so the interface has # …and the receiver binds that bridge address, so the interface has
# to exist. `exposeHostPorts` is a firewall hole, not a demand. # to exist. `exposeHostPorts` is a firewall hole, not a demand.
services.hyperhive.network.enable = lib.mkDefault true; services.hyperhive.network.enable = lib.mkDefault true;

14
nix/module-eval/core-toggle.nix
View file

@ -199,20 +199,6 @@ let
&& !(lib.hasInfix "127.0.0.1" e.HIVE_AGENT_NATS_URL) && !(lib.hasInfix "127.0.0.1" e.HIVE_AGENT_NATS_URL)
&& e.HIVE_AGENT_NATS_URL == e.HIVE_C0RE_NATS_URL; && e.HIVE_AGENT_NATS_URL == e.HIVE_C0RE_NATS_URL;
} }
{
# What agents are handed lands in their builds, so it has to be the same
# on every hive: a name, which this hive's resolver answers with the
# bridge IP its collector binds. hive-c0re itself keeps the address.
name = "the agents' collector address is a swarm-wide name this hive resolves to its bridge";
ok =
let
e = allLocal.systemd.services.hive-c0re.environment;
bridge = allLocal.services.hyperhive.network.bridgeIp;
in
e.HYPERHIVE_OTEL_ENDPOINT == "http://hive-collector.t.local:4318"
&& e.OTEL_EXPORTER_OTLP_ENDPOINT == "http://${bridge}:4318"
&& lib.elem "/hive-collector.t.local/${bridge}" allLocal.services.dnsmasq.settings.address;
}
{ {
# The agents mint against the swarm's IdP, the same endpoint the hive's # The agents mint against the swarm's IdP, the same endpoint the hive's
# own client uses — a hive-local guess would produce a token the queue # own client uses — a hive-local guess would produce a token the queue