vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

swarm-bao: stop linking the container's journal onto the host

Browse source 
--link-journal=host bind-mounts a host directory journald never
writes into for this container (empty, root:nogroup, confirmed on
the live host — #4527). Dropping it falls back to nixpkgs' default
--link-journal=try-guest, the same shape every agent container
already uses, so the in-container collector's journald receiver
(directory = /var/log/journal) now reads a journal that is actually
written.

merge stays true and services.hyperhive.swarm.otel.journaldUnits is
untouched so this deploy changes exactly one thing; comments that
described the old host-linked shape are rewritten to match.

Adds a module-eval assertion (bao-otel-collector.nix) that the
container's extraFlags never re-add --link-journal=host.

Refs #4527, #4499.
This commit is contained in:
atlas 2026-09-24 23:12:10 +02:00
commit 44009dc51d
2 changed files with 35 additions and 22 deletions
Download patch file Download diff file Expand all files Collapse all files

47
nix/host-modules/swarm-bao.nix
View file

@ -1645,9 +1645,13 @@ in
# collector (see `receivers.prometheus` below), which travels with the
# store.
#
# Logs are untouched by that move: `journaldUnits` above still names this
# store's units for the shared collector, and the two are retired
# together once every sibling swarm container has a collector of its own.
# Logs are a different story from that move: `journaldUnits` above still
# names this store's units, but bao's journal now lives only inside the
# container (see the forwarder's own comment below) — nothing links it
# into the host tree any more, so the shared collector can never see
# these units. The list stays untouched for the sibling containers that
# still ride it; bao's own entries come out once delivery through the
# container's own collector is confirmed.
# ⚠️ The one nginx exception to this file's header, and it is one because
# it never terminates. `ssl_preread` reads the SNI off the ClientHello
@ -2409,9 +2413,6 @@ in
containers.${cfg.machine} = {
autoStart = true;
ephemeral = false;
# Journal files on the host, not inside the container: nixpkgs hardcodes
# --link-journal=try-guest, and EXTRA_NSPAWN_FLAGS expands after it.
extraFlags = [ "--link-journal=host" ];
# Shared host netns, like every sibling swarm container. Unlike them the
# gateway is NOT the client here (see the no-vhost note at the top), so
# sharing the netns is what lets the store bind the host's own addresses
@ -2660,12 +2661,11 @@ in
# is supposed to ship its own logs to the next hop rather than
# leave a *different* container's collector to find them.
#
# That older shape is what `--link-journal=host` above and the
# `swarm.otel.journaldUnits` entry near the top of this file serve:
# the journal lands in the host tree, where the swarm collector —
# itself a container — reads it through a unit allowlist. Both stay
# for now, because every sibling swarm container still rides that
# collector; they come out once each of them has one of these.
# The journal lives inside this container now, the same shape as
# every agent container: nothing links it into the host's
# /var/log/journal tree, so `swarm.otel.journaldUnits` near the
# top of this file can no longer reach any of bao's units through
# the shared collector — see the comment there.
#
# ⚠️ NO `units` allowlist here, and that is the point rather than a
# simplification. A shared collector needs one because the journal
@ -2730,17 +2730,20 @@ in
receivers.journald = {
# ⚠️ STATED, and it must stay stated: the receiver's own
# default is the RUNTIME journal (`/run/log/journal`), and
# every entry here is under /var/log/journal — which
# `--link-journal=host` above makes the host's directory for
# this machine id. Dropping this line leaves a collector that
# validates, starts, reports healthy and forwards nothing.
# default is the RUNTIME journal (`/run/log/journal`), which
# in this container is empty — journald stores persistently
# here, so every entry is under /var/log/journal. Dropping
# this line leaves a collector that validates, starts,
# reports healthy and forwards nothing.
directory = "/var/log/journal";
# ⚠️ `--merge`, and it is what makes this receiver read
# anything at all: `journalctl --follow` scopes itself to the
# current boot, which has no entry under the host's journal
# tree — so every start died with "No journal boot entry found
# for the specified boot (+0)" in a ~2s crash-loop.
# ⚠️ `--merge` stays from when this journal was host-linked
# and `journalctl --follow` (unqualified) found no entry for
# the current boot under that tree, crash-looping the
# receiver every ~2s with "No journal boot entry found for
# the specified boot (+0)". The journal is local now, so the
# current boot has an entry here too — kept rather than
# dropped so the first deploy changes exactly one thing;
# revisit once delivery is confirmed.
merge = true;
storage = "file_storage";
};