otel: forward each agent container's journal to its hive collector

An agent container writes a complete journal — 991 MB and nine days deep
on this hive — that nothing outside it can read: the host-side
per-container journal directory is an id-mapped bind mount, and journald
writes nothing into it. So the reader has to run inside the container,
and the path it would push to did not exist.

Three tiers, one vertical slice, because any two of them alone are
silent:

- the agent container gains an `opentelemetry-collector` with a
  `journald` receiver aimed at its own journal and an exporter aimed at
  the same base address every in-process producer already exports to.
- the hive collector gains `service.pipelines.logs`. Without it the
  `otlp` receiver answers 404 on `/v1/logs` — measured, and
  indistinguishable from a route that was never meant to exist.
- the swarm collector gains a per-hive `logs/<hive>` pipeline beside
  `metrics/<hive>`. Without it the push is accepted, answered 200, and
  routed nowhere.

The receiver's `directory` is stated rather than inherited, and that is
the load-bearing line: its default is the RUNTIME journal
(`/run/log/journal`), which in an agent container is empty. Left at the
default this whole path validates, starts, reports healthy and forwards
nothing. The assertion beside it covers the same silence from the other
end — a `volatile` or `none` journald storage empties the directory the
receiver reads.

Attribution follows the tier that can prove it. The forwarder stamps
`agent`, which no host-side reader could supply; `hive` is deliberately
left to the swarm tier, which upserts it from whichever receiver
accepted the sample, precisely so the label comes from something the
sender cannot write.

No `units` allowlist, unlike the swarm tier's journald receiver. That
one needs one because the host's journal also holds an operator's own
session; a container's journal is the harness and what the harness
spawns. Measured volume is 20827 entries / 6.3 MB per agent per day,
with nothing logging below `info` — so the receiver's `info` default
filters nothing and there is no bill to justify a knob.

Agent containers only, per the ruling on the issue: swarm services need
one forwarder per service container and get re-measured once this works.

Part of #3940.

nix/host-modules/otel.nix

@@ -532,6 +532,30 @@ in
             ];
             exporters = [ swarmName ];
           };
+
+          # The second signal this tier carries, and the reason it exists
+          # is that an agent's journal is unreadable from here: the
+          # host-side per-container journal directory is an id-mapped bind
+          # mount journald never writes into, so each agent container runs
+          # its own forwarder (nix/agent-modules/otel.nix) and pushes over
+          # the `otlp` receiver above. Without this pipeline that receiver
+          # answers 404 on `/v1/logs` — measured, and indistinguishable
+          # from a route that was never meant to exist.
+          #
+          # One receiver, two pipelines: `otlp` accepts both signals and
+          # each pipeline takes the one it names. The exporter needs no
+          # change either — its `endpoint` is a base, and otlphttp appends
+          # `/v1/logs` to it exactly as it appends `/v1/metrics`.
+          service.pipelines.logs = {
+            receivers = [ "otlp" ];
+            # `resourcedetection` and NOT `deltatocumulative`: that one is
+            # a metrics-only processor, and naming it here is a startup
+            # failure rather than a no-op. Host identity applies to both
+            # signals for the same reason — a line is only comparable
+            # across hives once it says which machine collected it.
+            processors = [ "resourcedetection" ];
+            exporters = [ swarmName ];
+          };
         }
         // lib.optionalAttrs senderAuth {
           extensions.${authName} = {

nix/host-modules/swarm-otel.nix

@@ -1255,6 +1255,28 @@ in
                     exporters = exporterNames;
                   };
                 }
+                # A hive's logs, and the receiver is the SAME one its metrics
+                # arrive on: a receiver is not per-signal, so `otlp/${h}`
+                # feeds this pipeline and `metrics/${h}` above with no second
+                # port, credential or audience. Which is also why the `hive`
+                # label stays trustworthy for logs without any new mechanism
+                # — it still comes from which receiver accepted the sample.
+                #
+                # The senders are agent containers, one forwarder each
+                # (nix/agent-modules/otel.nix), via their own hive's collector.
+                # Without this pipeline that whole path terminates here: the
+                # receiver accepts the push, answers 200, and the records go
+                # nowhere.
+                // lib.optionalAttrs collectLogs (
+                  lib.mapAttrs' (
+                    h: _:
+                    lib.nameValuePair "logs/${h}" {
+                      receivers = [ "otlp/${h}" ];
+                      processors = [ "resource/${h}" ];
+                      exporters = logExporterNames;
+                    }
+                  ) hivePorts
+                )
                 # Logs fan out exactly as metrics do: the swarm's store when it
                 # runs, the operator's upstream when one is configured, both
                 # when both. The local store is a destination rather than the