A set HYPERHIVE_CLAUDE_CODE_PATH whose build is gone (collected after a
bump while the container still holds the old facts) made the wrapper
run the container's own claude-code silently, which can be a different
version. It now prints one line naming the missing path.
Refs #4857
The three values that differ per host (deploy.hive-controller.dashboardPort,
claudeCodePackage and otel.extraResourceAttributes) no longer go into the
meta flake. hive-priv writes them into the container's
/etc/hyperhive-hive-facts with the hive name and domain, and the agent
reads them at boot:
- HIVE_DASHBOARD_PORT reaches every unit through the hive-facts generator;
the web UI still defaults to 7000 without it.
- `claude` on PATH is a wrapper that runs HYPERHIVE_CLAUDE_CODE_PATH's
claude when it is there, and the container's own claude-code otherwise.
The host's serve.json keeps gc-rooting the pinned build.
- OTEL_RESOURCE_ATTRIBUTES moves out of the units' Environment= into a
boot-time generator, which appends HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES.
The agent options services.hyperhive.agent.claudeCodePath (now a removed
option) and otel.extraResourceAttributes go; only the meta render set them.
render_flake takes no HiveEnv any more, so the memory-cap render test and
render_hive_flake_at, which only existed to vary HiveEnv into the render,
go with it.
Refs #4857
`swarm-bao-agent-pki` carried its own copy of the mtime-vs-start compare,
with the same `stat -c %.6Y | tr -d .` read: on a comma-locale host the
`-le` errored and, under `seal = "pkcs11"`, openbao was restarted on
every run of the unit.
The compare moves out of `refresh_unit` into `written_after_start`
(0 newer, 1 not newer, 2 unreadable and logged at `err`), and both
`refresh_unit` and the agent-pki unit call it. The agent-pki unit cannot
call `refresh_unit` itself: it restarts with `restart`, not
`try-restart`, runs a fail-closed check of the bundle between the
compare and the restart, and under shamir prints the step instead of
restarting. It acts only on 0, so an unreadable time skips the restart
as `refresh_unit` does.
`script-test-refresh-consumer` covers the three return codes in the
agent-pki call shape; `module-eval-bao-basics` asserts both seal arms
decide on that call and that no `%.6Y` read is left in the unit.
Refs #5230
`refresh_unit` read the credential file's mtime with
`stat -c %.6Y | tr -d .`. coreutils prints that fraction with the
locale's decimal separator, so on a host whose locale uses a comma
(muede-lpt2: `1791232888,193159`) `tr` stripped nothing, the `-le`
against the unit's start errored, the "unchanged" early return was
skipped, and every run try-restarted the consumer. Every unit on a
store-refresh timer was restarted every ~10 minutes; for
`swarm-bao-ci-runner` that is `gitea-runner-swarm`, which killed every
CI job running at the time.
The mtime is now read with `date -r … +%s%6N`, the same digit-only
format the start time already used (`date -d … +%s%6N` on systemd's
fixed English timestamp), so neither side has a separator for a locale
to change.
A value that still does not read as digits — an unparsable start time,
a missing file — now restarts nothing and logs at `err`. Skipping on
doubt rather than restarting on doubt: the file's mtime is kept, so a
genuinely changed credential still restarts its consumer on the first
run that can read both times, whereas restarting on doubt restarts it
on every run, which is this bug. Both reads also fall back to empty
instead of aborting the caller's `set -e` script.
`script-test-refresh-consumer` runs the rendered helper under
`de_DE.UTF-8` against a stub `systemctl`, and checks first that `stat`
really prints a comma there. The helper on main fails 14 of its
assertions.
Fixes#5230
The cap file was removed for a cap of `infinity` or a RAM percentage,
so the boot check warned on every start of such an agent, for a
legitimate config. The file is now written on every start:
- a size: its byte count, as before;
- `infinity`: the literal `infinity`. The generators set no JSC heap
ceiling and no subagent MemoryHigh=, and the boot check is quiet;
- a percentage: resolved on the host the way systemd resolves
MemoryMax=N% for the container@ unit. That is permyriad (at most two
decimals, at most 100%) of sysconf(_SC_PHYS_PAGES) *
sysconf(_SC_PAGESIZE), rounded down to a page;
- a value systemd ignores (zero, over 100%, more than two decimals,
unparseable) leaves the unit uncapped, so it is `infinity`.
The file is missing only when hive-c0re can't tell the cap: the
limits file and the kept drop-in are both unreadable, or a percentage
meets an unknown host memory size. The boot check now warns only for
a missing file or a first line that is neither a byte count nor
`infinity`.
Part of #5081 (slice S0-2).
When hive-c0re can't read resource-limits.json and a limits drop-in
already exists, it keeps the drop-in, but it wrote no memory cap file. A
container whose runtime dir had no file yet (hive-c0re upgraded, /run
not cleared) then started with no heap ceiling and no subagent
MemoryHigh=, silently. `stage_limits` now writes the file on every
call: from the new limits, or from the last `MemoryMax=` in the kept
drop-in (`infinity` when it has none or can't be read).
A missing or malformed cap file at boot is no longer silent. The
generators run before the container's journal and an nspawn container
has no /dev/kmsg, so a oneshot `hyperhive-memory-cap-check.service`
logs a warning naming the path. The agent then runs with no JSC heap
ceiling and no subagent MemoryHigh=; the container's MemoryMax= still
applies.
module-eval-agent-memory gains a case pinning how the ceiling reaches
claude: `hive-agent` and `hive-subagent-daemon` neither set nor unset
BUN_JSC_forceRAMSize, both generators and the check unit are installed.
Part of #5081 (slice S0-2).
An agent's memory cap was rendered into its flake (`memoryMaxBytes` →
`services.hyperhive.agent.claudeMemoryMaxBytes`), so the closure depended
on it and a changed cap reached the JSC heap ceiling and the subagent
`MemoryHigh=` only after a rebuild.
hive-c0re now writes the effective `MemoryMax=` in bytes to the agent's
runtime dir (`/run/hive/memory-max-bytes` inside) in `write_dropins`,
from the same value the drop-in gets. The file is removed for
`infinity` or a RAM percentage, and left alone when the drop-in is kept.
Inside the container, `nix/agent-modules/memory-cap.nix` reads it at
boot: an environment generator sets `BUN_JSC_forceRAMSize` (75%) for
every unit and `machinectl shell`, `/etc/set-environment` exports it in
login shells, and a unit generator writes `MemoryHigh=` (2/3) as a
drop-in for `hive-subagent-daemon`.
The render drops `memoryMaxBytes`, the agent option is removed (both
spellings fail eval with a removal message), and
`effective_memory_bytes_from` goes with its only caller.
Part of #5081 (slice S0-2).
Part of #5081. Follow-up to the review of #5131.
- Before this PR, claude read CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX
from managed-settings.json in every context. The environment generator
reaches every unit of the container's manager, including `machinectl
shell` (hivectl choom): machined starts it as a transient
container-shell@ unit (systemd 260.5 src/machine/machine.c:980-1004),
and every unit's exec environment starts from the manager environment
the generators fill (src/core/unit.c:5557, src/core/manager.c:3990-4001).
Shells no unit starts do not get it: `nixos-container root-login` / `run`
enter with nsenter + `su -l`, and getty logins go through login(1). So
hive-facts.nix now also exports the generator's output from
/etc/set-environment (environment.extraInit).
- The generator's read loop kept a last line only when it ended in a
newline; it now keeps one without.
- checks.script-test-agent-hive-facts runs the rendered generator and the
shell snippet (under dash and bash's sh) against sample files: both
facts, a planted LD_PRELOAD/PATH line, a last line without a newline,
an empty file and no file.
- identity.rs: hive_name()'s doc names the facts file as the source.
The live pair was replaced with two renames, key then cert. A run
stopped between them (timeout, container stop, power loss) left a new
key beside the old cert, and every later run logged in with that pair,
so the agent could not re-pull until its container restarted.
Each pair now goes into a fresh directory under
/run/hive-agent-bao-identity, and `current`, a symlink to it, is
replaced by one rename. Wherever a run stops, `current` names one whole
pair; the next run removes whatever a stopped run left. Consumers read
`current/cert` and `current/key` through the same read-only options and
still re-read them on every login.
The script test covers a swap stopped at each of its link and rename
steps: the live pair is always one whole pair, and the next run puts
the new one live.
Refs #5017
swarm-controller re-issues an agent's store leaf at half its validity, but
the running container kept presenting the copy its hive delivered at
start, so a re-issued leaf reached the agent only on its next restart.
hive-agent-bao-identity now seeds a live copy of the pair in its own
runtime directory from the delivered credentials, and re-runs every 10
minutes: it logs in with the live leaf, reads its own
swarm/agents/<agent>/bao-mtls (a read render_agent already grants), and
when the leaf changed, logs in with the new pair and swaps it in only if
the store accepts it.
Every unit in the container that logs in to the store presents the live
pair: hive-agent (queue-identity.nix), hive-matrix-daemon,
hive-subagent-daemon, hive-agent-forge-token, hive-agent-forge-accounts
and hive-agent-github-token. They keep loading the server CA as a
credential. The long-running daemons are ordered after the refresher so
the live pair exists when they start.
It renews only while the held leaf is still valid: the login that reads
the new leaf is made with the current one.
Refs #5017
`swarm-bao-nats-callout-xkey` runs `ensure-callout-keys`, importing
/var/lib/swarm-nats-callout/{issuer,callout-user}.seed when present so the
keys keep their public halves, then fetches all three public halves from
the store. The verb removes each seed file once a read of the store returns
a seed; when the store's seed differs, it is kept and a warning names the
file. No tmpfiles rule touches those files, since one could run before the
import with the store sealed and leave the next run to mint a new key.
The server renders the issuer and user public halves from the fetched files,
checked for shape as the xkey's is, and does not start without them. The
responder reads both seeds from the store, so the copies into the container
and their LoadCredential entries are gone, and a tmpfiles rule removes the
copies a container still holds. `swarm-nats-callout-keys` is removed.
On the store's host the fetch unit is also ordered after
`swarm-bao-secret-publisher-policy`, so the publisher's denial of the seed
paths is written before the unit creates or keeps a seed there.
`deploy.nats.autoGenerateCallout`, `calloutUserPublicKey`,
`calloutIssuerPublicKey`, `calloutUserSeedFile` and `calloutIssuerSeedFile`
are removed with `mkRemovedOptionModule`, under `swarm.nats` too, each
message saying the keys live in the store.
`checks.script-test-nats-render-config` runs the queue server's
`ExecStartPre` render against a scratch runtime directory: a fetched key
lands in `auth_callout.xkey` and the wrapper includes the fragment, and
a missing or malformed key file fails the start and writes no config.
module-eval only sees the render as a store path, so dropping `xkey`
from the template, or its substitution, passed every check while the
server would have run unsealed.
The harness now reads swarm/agents/<agent>/queue from the store itself,
under the agent's own store certificate, and holds it in memory only.
It reads once before the first connect and again on every reconnect
attempt (async-nats `ConnectOptions::with_auth_callback`), so an agent
whose secret was re-minted reconnects with the new value instead of
being refused until the container restarts.
hive-agent-queue-credential.service, the /run file it wrote, and
HIVE_AGENT_QUEUE_AGENT_SECRET_FILE are gone; queue-identity.nix now
hands hive-agent.service the store address, its certificate paths and
the agent name.
A failed or empty read before the first connect still falls back to
the hive's shared client. Each read is bounded by a 10s timeout, and
retries wait out the existing reconnect backoff (500ms doubling, capped
at 60s).
Closes#4783
`script-test-agent-bao-fetch` executes the rendered `ExecStart` of
`hive-agent-forge-token` and `hive-agent-queue-credential` under
`umask 0377`, with a stub `bao` first on the unit's own PATH. It covers
every error branch, the happy path, forge rotation/unchanged, and 0400
files already in place: the redirect failure #4736 fixed, whose live
symptom was `bao.err: Permission denied` reported as a refused
certificate. The TLS-alert fixture is the `unknown certificate
authority` error h-atlas's identity check got from the store.
#4736 claimed this test but never committed it; the two module-eval
suites for these units only evaluate the config.
Closes#4748
