a39399f037
An agent can reach VictoriaLogs only through the gateway, and since the
machine query route landed the way to read it has been to hand-roll a
client_credentials token request and a curl, per query. This is the CLI
that closes that: `swarm-logs query '<LogsQL>'`, matched log lines on
stdout, so the answer pipes into grep like any other command's.
Built to the plan posted on the tracker thread: own crate, own
docs/tools reference generated off the clap tree, `query` as the one
verb, and the JSON error body surfaced on a non-200 rather than
swallowed. No `tail`: streaming is a different endpoint with a different
response shape, and folding it in here would be a fatter scope than the
ask.
Minting the token is NOT implemented here — swarm-queue-client already
owns the client_credentials request, its error type and its CA handling,
and a token-endpoint fix has to be findable in one place. What this crate
adds is the agent-shaped half: the client id arrives as a *file* beside
the secret, so nothing outside nix/agent-modules/queue.nix spells
`hive-<name>-agent` twice. That is the same problem hive-agent's
swarm_queue module solves, and swarm-logs/src/auth.rs is its `decide`
restated over this binary's inputs.
⚠️ The plan named one thing to verify empirically before calling the auth
settled: whether authelia's bearer policy for the logs vhost accepts the
agent client's audience. Measured from inside a container: it does not.
The client minted a token fine but with `aud: []` and `scp: []`, asking
for the logs URL as an audience answered `invalid_target`, and presenting
the audience-less token to the gateway answered a bare 401. So
swarm-authelia.nix's agentClients gains `authelia.bearer.authz` and the
query URL as a second audience — authelia authorises a bearer token by
the URL being requested, and that URL is now one binding read by three
places rather than three spellings of one address.
The URL reaches an agent the same way its queue coordinates do: computed
on the host (a container cannot derive a gateway address), forwarded by
hive_c0re::meta into the container's option set, and consumed by a new
agent module that installs the binary *wrapped* with its coordinates —
the shape swarm-controller.nix installs swarmctl in. Gated on the queue
credential as well as on the URL: a binary that can only answer 401 is
worse than no binary, because an agent reads a 401 as "no logs", which is
the exact confusion the store's machine route was added to end.
348 lines
18 KiB
Nix
348 lines
18 KiB
Nix
# Environment of the hive-c0re daemon unit — a plain function file
|
|
# (not a module) returning the env attrset, imported by ./default.nix.
|
|
# Everything meta.rs forwards into agent containers or reads for the
|
|
# meta-flake render is assembled here.
|
|
{
|
|
lib,
|
|
config,
|
|
pkgs,
|
|
}:
|
|
let
|
|
cfg = config.services.hyperhive.c0re;
|
|
baoDeploy = config.services.hyperhive.deploy.bao;
|
|
baoCfg = config.services.hyperhive.swarm.bao;
|
|
# The same test ../glue-matrix-bao-token.nix applies, and for the same
|
|
# reason: a reader is defined by holding a certificate the store accepts,
|
|
# not by sharing a host with the store. Deriving this from
|
|
# `deploy.bao.enable` would be the co-location assumption itself.
|
|
haveBaoClientIdentity = baoDeploy.clientCertFile != null && baoDeploy.clientKeyFile != null;
|
|
in
|
|
{
|
|
# nix (the prebuild `nix build`, flake-check, and meta eval) writes
|
|
# its fetcher/eval cache under $HOME/.cache. As a system user
|
|
# hive-core has no home, so HOME defaults to the unwritable
|
|
# /var/empty and Lix fails to initialise its cache. Point HOME at
|
|
# the writable StateDirectory.
|
|
HOME = "/var/lib/hyperhive";
|
|
HYPERHIVE_GIT = "${pkgs.git}/bin/git";
|
|
# Never let git block on an interactive credential prompt. hive-core is a
|
|
# TTY-less system user, so a prompt (e.g. the forge credential helper
|
|
# returns nothing because the forge isn't reachable yet on cold boot)
|
|
# would hang forever — this is what froze the whole daemon during startup
|
|
# migration's `nix flake lock` of the forge-hosted config inputs. With
|
|
# this set, git fails fast instead of prompting. Paired with the git http
|
|
# low-speed abort in `safeDirGitconfig` (bounds a stalled transfer) and
|
|
# the 120s migration shellout timeout in migrate.rs.
|
|
GIT_TERMINAL_PROMPT = "0";
|
|
# No HIVE_STATIC_DIR: the gateway static-serves the dashboard dist
|
|
# (see the hive-gateway module); this router is API-only.
|
|
# Path to the base agent frontend dist. hive-c0re's
|
|
# gateway_nginx.rs uses this to generate split location
|
|
# blocks in agents.conf — static HTML/CSS/JS served from the
|
|
# nix store directly; dynamic API paths still proxied to the
|
|
# agent daemon. nginx runs on the host, which is where this
|
|
# store path already is, so it is reachable as written.
|
|
HIVE_AGENT_FRONTEND_DIR = "${cfg.servedFrontend}/agent";
|
|
# Path to the static runtime asset tree (branding + claude
|
|
# prompts). `hive_sh4re::assets::*` reads paths underneath.
|
|
# `forge/users.rs` reads the core avatar PNG from here on startup.
|
|
HIVE_ASSETS_DIR = "${cfg.assets}/share/hyperhive";
|
|
# `agent-configs` org avatar PNG — independently overridable via
|
|
# `orgAvatarPng` without replacing the whole `assets` package.
|
|
# Falls back to the bundled PNG under HIVE_ASSETS_DIR when unset.
|
|
# Read by `forge::users::config_org_avatar_png_path`.
|
|
HIVE_ORG_AVATAR_PNG =
|
|
if cfg.orgAvatarPng != null then
|
|
"${cfg.orgAvatarPng}"
|
|
else
|
|
"${cfg.assets}/share/hyperhive/branding/agent-configs.png";
|
|
# Whether this hive runs ruthless — no root/manager agent at all
|
|
# (`auto_update::ensure_root_agent`). Default false = root
|
|
# auto-managed; true makes the sweep a no-op.
|
|
HYPERHIVE_RUTHLESS = lib.boolToString config.services.hyperhive.ruthless;
|
|
}
|
|
// {
|
|
# Identity env vars threaded into c0re's own service env and
|
|
# forwarded by meta.rs into every sub-agent's harness env —
|
|
# full chain in docs/process/conventions.md::Hive identity. `domain` is
|
|
# required (asserted in hive-network.nix), so it's always set.
|
|
HYPERHIVE_HIVE_DOMAIN = config.services.hyperhive.domain;
|
|
}
|
|
// lib.optionalAttrs (config.services.hyperhive.hiveName != null) {
|
|
HYPERHIVE_HIVE_NAME = config.services.hyperhive.hiveName;
|
|
}
|
|
// lib.optionalAttrs (config.services.hyperhive.swarm.name != null) {
|
|
HYPERHIVE_SWARM_NAME = config.services.hyperhive.swarm.name;
|
|
}
|
|
// lib.optionalAttrs (!config.services.hyperhive.github.enable) {
|
|
# GitHub integration is on by default; only signal the OFF override to
|
|
# meta.rs, which then injects `hyperhive.github.enable = false` into
|
|
# every agent. See services.hyperhive.github.enable.
|
|
HYPERHIVE_GITHUB_DISABLED = "1";
|
|
}
|
|
// lib.optionalAttrs config.services.hyperhive.otel.enable (
|
|
# Hive-wide OTEL config -> read by meta.rs::otel_config and
|
|
# injected as build-time `hyperhive.otel.*` into every agent.
|
|
# Endpoint presence is the enable signal on the meta side; the
|
|
# optional fields are only emitted when set so absent values
|
|
# don't render no-op env lines.
|
|
let
|
|
otel = config.services.hyperhive.otel;
|
|
# The first hop, bound once and consumed twice below: what agents are
|
|
# handed, and where hive-c0re's own exporter sends. One binding so the
|
|
# address a hive tells its agents about and the one it uses itself
|
|
# cannot drift apart.
|
|
firstHop = "http://${config.services.hyperhive.network.bridgeIp}:${toString otel.collector.port}";
|
|
in
|
|
{
|
|
# `otel.endpoint` means "where telemetry ultimately goes" and keeps
|
|
# that meaning; what agents are handed is the *first hop*, which is
|
|
# always this hive's own collector. Deriving it rather than
|
|
# redefining `endpoint` is what lets every existing deployment keep
|
|
# its configured value untouched.
|
|
HYPERHIVE_OTEL_ENDPOINT = firstHop;
|
|
# hive-c0re's OWN container-resource exporter (stats/otel_metrics.rs)
|
|
# reads the STANDARD OTLP variable — the same one hive-metric and every
|
|
# agent read — and lets the SDK resolve the URL, which appends the
|
|
# signal path (`/v1/metrics`). Handing the SDK an address
|
|
# programmatically instead takes it verbatim: it POSTs to the
|
|
# collector's root, gets a 404 on every export, and says nothing,
|
|
# because OTLP export failures go to an error handler no binary here
|
|
# installs — the daemon logged "exporter enabled" and never delivered a
|
|
# sample, for as long as it had that exporter. The variable above cannot
|
|
# replace this one:
|
|
# it is the agent-config transport meta.rs reads, and the SDK does not
|
|
# know that name.
|
|
OTEL_EXPORTER_OTLP_ENDPOINT = firstHop;
|
|
# The first hop is the collector's OTLP/HTTP receiver, which speaks
|
|
# protobuf regardless of what the upstream wants — `otel.protocol`
|
|
# describes the *upstream* link, and the collector's own exporter is
|
|
# what has to honour it (see nix/host-modules/otel.nix).
|
|
HYPERHIVE_OTEL_PROTOCOL = "http/protobuf";
|
|
}
|
|
// lib.optionalAttrs (otel.extraResourceAttributes != "") {
|
|
HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES = otel.extraResourceAttributes;
|
|
}
|
|
# HYPERHIVE_OTEL_HEADERS_CREDENTIAL is deliberately NOT emitted, and
|
|
# its absence is the security half of this design: it was the variable
|
|
# that put the upstream token into an agent's own settings.json. The
|
|
# delivery path it drove — an nspawn credential forwarded by
|
|
# host_config.rs, then written to an agent-readable file by
|
|
# claude-settings.nix's `hive-otel-header` oneshot — no longer exists
|
|
# anywhere; it was removed along with this variable's last consumer.
|
|
#
|
|
# Kept as a comment rather than deleted because the useful part is the
|
|
# RULE, not the history: the collector holding the credential achieves
|
|
# nothing while anything else hands out a copy, so there is exactly one
|
|
# holder and it is on the host.
|
|
// lib.optionalAttrs (otel.metricIntervalMs != null) {
|
|
HYPERHIVE_OTEL_METRIC_INTERVAL_MS = toString otel.metricIntervalMs;
|
|
}
|
|
// lib.optionalAttrs otel.debug {
|
|
HYPERHIVE_OTEL_DEBUG = "1";
|
|
}
|
|
)
|
|
// {
|
|
# In-cluster forge URL — the gateway vhost (`forge.<domain>`), which
|
|
# nginx proxies to forgejo. Used both for internal API calls in
|
|
# hive-c0re (forge/mod.rs `forge_http_base()`) and forwarded to
|
|
# agents via meta.rs for their forge-notify client. The forge is
|
|
# mandatory, so this is unconditional (the whole env block is already
|
|
# gated on hyperhive being enabled). See `docs/networking/gateway.md::HIVE_FORGE_URL`.
|
|
HIVE_FORGE_URL = "http://${config.services.hyperhive.swarm.forge.domain}";
|
|
|
|
# The one blacklist of names an agent may not take — see
|
|
# `nix/reserved-names.nix`, which is also read by the swarm controller, by
|
|
# the swarm collector's owner assertion, and by the test suite. Nix owns it
|
|
# so that keeping it current is a config change, not a rebuild of a binary.
|
|
#
|
|
# Whitespace-separated rather than JSON: every entry is an `Ident`
|
|
# (`[a-z0-9-]`), so a space can never occur inside a name and the encoding
|
|
# cannot be lossy. Spelling it as JSON would put a parser in the crate whose
|
|
# whole point is to have no dependencies.
|
|
#
|
|
# Unconditional on purpose. The consumer treats an ABSENT variable as "I was
|
|
# never told" and says so out loud, which is the correct reading — but it is
|
|
# a reading no correctly-built hive should ever have to make.
|
|
HIVE_RESERVED_NAMES = lib.concatStringsSep " " (import ../../reserved-names.nix);
|
|
}
|
|
// lib.optionalAttrs (config.services.hyperhive.swarm.matrix.gatewayHost != null) {
|
|
# Matrix homeserver URL for each agent's hive-matrix-daemon — the
|
|
# gateway vhost (`chat.<swarm-domain>`). Forwarded to agents by meta.rs
|
|
# alongside HIVE_FORGE_URL; shares the same env-forwarding ordering
|
|
# caveat (value baked at config-generation time).
|
|
#
|
|
# Deliberately NOT conditioned on this host running the homeserver. A
|
|
# swarm has one matrix, and every hive's agents talk to it — the hive
|
|
# that hosts it is not the only hive whose agents need its address.
|
|
# `gatewayHost` is swarm-scoped and names that one homeserver from any
|
|
# member hive, so its being set is the whole question.
|
|
#
|
|
# Null forwards nothing rather than falling back to loopback. That
|
|
# fallback reads as harmless because hive-c0re shares the host netns —
|
|
# but the value is handed to *agents*, which do not, so `127.0.0.1`
|
|
# there names the agent itself. An absent forward leaves
|
|
# `hyperhive.matrix.url` null and the daemon no-ops; that is the honest
|
|
# answer when there is no matrix vhost to point at.
|
|
HIVE_MATRIX_URL = "http://${config.services.hyperhive.swarm.matrix.gatewayHost}";
|
|
}
|
|
// lib.optionalAttrs (config.services.hyperhive.swarm.matrix.apiUrl != null) {
|
|
# Client-server API base hive-c0re uses to provision matrix (register
|
|
# agent users, create the hive space + chat room, invite members).
|
|
# Supplied by `services.hyperhive.swarm.matrix.apiUrl`, which the matrix
|
|
# module fills in with its own loopback listener when it is the thing
|
|
# running tuwunel — and which the operator sets by hand when the
|
|
# homeserver lives on another machine.
|
|
#
|
|
# NOT the agent-facing HIVE_MATRIX_URL above: that one is the gateway
|
|
# vhost, and it is absent whenever there is no vhost. Reusing it here
|
|
# would silently stop provisioning on a hive that runs matrix without
|
|
# one.
|
|
HIVE_MATRIX_API_URL = config.services.hyperhive.swarm.matrix.apiUrl;
|
|
}
|
|
// lib.optionalAttrs config.services.hyperhive.deploy.matrix.gui.enable {
|
|
# Availability flags read by the dashboard's `/api/state`.
|
|
# Matrix GUI lives entirely on the gateway nginx (matrix tab
|
|
# only shows when both flags are on). Gateway routing detail:
|
|
# docs/networking/gateway.md::Vhost map.
|
|
HIVE_MATRIX_GUI_ENABLED = "1";
|
|
}
|
|
// {
|
|
# The gateway always runs, so the dashboard always builds
|
|
# same-origin `/agent/<name>/` links (never the direct
|
|
# `<host>:<port>` TCP fallback). Kept as an env flag so the
|
|
# dashboard doesn't need to learn the gateway is unconditional.
|
|
HIVE_GATEWAY_ENABLED = "1";
|
|
}
|
|
// lib.optionalAttrs (config.services.hyperhive.swarm.forge.publicUrl != null) {
|
|
# Public URL of the forge, for the dashboard to build browser-facing
|
|
# forge links from instead of guessing `<hostname>:3000` (which
|
|
# breaks the moment the operator's browser hostname isn't the forge
|
|
# host, e.g. through the gateway or a reverse proxy). Sourced from
|
|
# `services.hyperhive.swarm.forge.publicUrl`, which itself defaults to the
|
|
# gateway vhost URL when `deploy.forgejo.behindGateway = true` and `null`
|
|
# otherwise — see that option's doc for the "hide, don't guess" rationale.
|
|
# Absent here whenever `publicUrl` is `null`; the dashboard hides
|
|
# forge links rather than emitting one it can't justify.
|
|
HIVE_FORGE_PUBLIC_URL = config.services.hyperhive.swarm.forge.publicUrl;
|
|
}
|
|
//
|
|
lib.optionalAttrs
|
|
(
|
|
config.services.hyperhive.deploy.matrix.gui.enable
|
|
&& config.services.hyperhive.swarm.matrix.gatewayHost != null
|
|
)
|
|
{
|
|
# Browser-facing matrix GUI (fluffychat) URL — the gateway
|
|
# vhost (`gatewayHost`, `chat.<swarm-domain>` by default). Surfaced via the daemon's `Urls`
|
|
# request for `hivectl open matrix`. Absent when the GUI is off
|
|
# or no gatewayHost is set (no browser-reachable matrix vhost).
|
|
HIVE_MATRIX_PUBLIC_URL = "https://${config.services.hyperhive.swarm.matrix.gatewayHost}/";
|
|
}
|
|
// lib.optionalAttrs (config.services.hyperhive.swarm.snapshotStore.address != null) {
|
|
# `host:port` of the swarm's single snapshot store, for pushing agent
|
|
# snapshots (hive-c0re::snapshot_push). One per swarm, not one per
|
|
# peer — the receiver keys destinations by agent so a migrating agent
|
|
# keeps one incremental chain. Absent when no store is configured, and
|
|
# a push then fails naming the option rather than guessing.
|
|
HYPERHIVE_SNAPSHOT_STORE =
|
|
let
|
|
s = config.services.hyperhive.swarm.snapshotStore;
|
|
in
|
|
"${s.address}:${toString s.port}";
|
|
}
|
|
//
|
|
# Swarm-queue coordinates for offering this hive's status upward
|
|
# (hive-c0re::swarm_status). All four together or none: a half-set
|
|
# environment is a deployment bug the daemon refuses to treat as
|
|
# "no queue configured", because the failure it would otherwise
|
|
# produce is a hive that comes up fine and silently never reports.
|
|
# The three-option version of that same rule is asserted at eval in
|
|
# ./../swarm.nix, so this can only ever emit a complete set.
|
|
#
|
|
# ⚠️ The guard and the value beside it read different namespaces on
|
|
# purpose: where the queue is and where its secret sits are this
|
|
# machine's, the token endpoint is the swarm's one address. The
|
|
# assertion covers all three, which is what keeps the guard honest.
|
|
lib.optionalAttrs (config.services.hyperhive.deploy.hive-controller.statusPublish.natsUrl != null) {
|
|
HIVE_C0RE_NATS_URL = config.services.hyperhive.deploy.hive-controller.statusPublish.natsUrl;
|
|
HIVE_C0RE_OIDC_TOKEN_ENDPOINT = config.services.hyperhive.swarm.statusPublish.tokenEndpoint;
|
|
# The identity swarm-authelia.nix already declares for every entry in
|
|
# `swarm.hives` — the hive does not choose its own name here, it uses
|
|
# the one the roster gave it.
|
|
HIVE_C0RE_OIDC_CLIENT_ID = "hive-${config.services.hyperhive.hiveName}";
|
|
# `%d` is systemd's credentials directory — see the LoadCredential in
|
|
# ./default.nix. The daemon reads a path, never a value.
|
|
HIVE_C0RE_OIDC_CLIENT_SECRET_FILE = "%d/swarm-status-client.secret";
|
|
}
|
|
// {
|
|
# Where ../glue-queue-agent-credential.nix lands the AGENTS' queue
|
|
# credential. Read by `hive_c0re::lifecycle::host_config`, which stats the
|
|
# two files and forwards them into each container as systemd credentials.
|
|
# Never read for its contents here: the secret is `0600` root-owned and
|
|
# this daemon is `hive-core`, which is exactly why the transport is a
|
|
# credential rather than a bind mount.
|
|
HIVE_C0RE_AGENT_QUEUE_CREDENTIAL_DIR = toString config.services.hyperhive.deploy.hive-controller.queue.agentCredentialDir;
|
|
}
|
|
//
|
|
# The agents' half of the same queue, forwarded by `hive_c0re::meta` into
|
|
# every container. A pair, gated together, because an agent that got one of
|
|
# them would report a half-configured queue instead of none.
|
|
#
|
|
# ⚠️ The url is the bridge address, NOT the loopback one beside it in
|
|
# `HIVE_C0RE_NATS_URL` above. Both are correct for their reader: this daemon
|
|
# shares the host netns, an agent does not, and inside a container
|
|
# `127.0.0.1` is the agent itself.
|
|
lib.optionalAttrs
|
|
(
|
|
config.services.hyperhive.deploy.hive-controller.queue.agentNatsUrl != null
|
|
&& config.services.hyperhive.swarm.statusPublish.tokenEndpoint != null
|
|
)
|
|
{
|
|
HIVE_AGENT_NATS_URL = config.services.hyperhive.deploy.hive-controller.queue.agentNatsUrl;
|
|
HIVE_AGENT_OIDC_TOKEN_ENDPOINT = config.services.hyperhive.swarm.statusPublish.tokenEndpoint;
|
|
}
|
|
//
|
|
# Where an agent reads the swarm's logs, forwarded the same way and gated on
|
|
# the same credential: `swarm-logs` authenticates as the queue's own per-hive
|
|
# machine client, so an address with no credential behind it would put a
|
|
# binary on PATH that can only ever answer 401 — which an agent reads as "no
|
|
# logs", the exact confusion the store's machine route was added to end.
|
|
#
|
|
# The **machine** route (`^~ /select/logsql/`), not the browser one at `/`:
|
|
# that one ends in `error_page 401 =302` and hands a machine caller
|
|
# authelia's login page as a 200 with an HTML body. See
|
|
# `swarm-victorialogs.nix`'s location for the whole reasoning.
|
|
#
|
|
# ⚠️ The full URL rather than the domain, because the same string is also
|
|
# the audience the token is minted for — `swarm-otel.nix` states that rule
|
|
# over its own push targets, and two spellings present as a valid token
|
|
# refused at the store.
|
|
lib.optionalAttrs
|
|
(
|
|
config.services.hyperhive.deploy.hive-controller.queue.agentNatsUrl != null
|
|
&& config.services.hyperhive.swarm.statusPublish.tokenEndpoint != null
|
|
)
|
|
{
|
|
HIVE_AGENT_LOGS_QUERY_URL = "https://${config.services.hyperhive.swarm.victorialogs.domain}/select/logsql/query";
|
|
}
|
|
//
|
|
# Where the swarm's secret store is, and the identity this hive presents to
|
|
# it (hive-c0re::workers::credential). `swarm_secret_client` reads these
|
|
# spellings explicitly rather than vaultrs's `VAULT_*` defaults — falling
|
|
# through to those would build a client with no identity and fail at the TLS
|
|
# handshake, naming neither.
|
|
lib.optionalAttrs haveBaoClientIdentity {
|
|
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
|
|
# `%d`, not the paths themselves: the key is `0600` root-owned and this
|
|
# daemon runs as hive-core, so it never gets read access to the original.
|
|
# See the LoadCredential in ./default.nix.
|
|
BAO_CLIENT_CERT = "%d/bao-client.pem";
|
|
BAO_CLIENT_KEY = "%d/bao-client-key.pem";
|
|
}
|
|
// lib.optionalAttrs (haveBaoClientIdentity && baoDeploy.serverCaFile != null) {
|
|
# Absent means the system trust store — right for a deployment with a real
|
|
# CA, wrong for the self-signed one ../glue-bao-tls.nix mints, which is why
|
|
# that file names this path rather than leaving it to a default.
|
|
BAO_CACERT = "%d/bao-ca.pem";
|
|
}
|