The gateway container's /etc/resolv.conf is a one-shot copy: nixos-container
cps it in from the host in its start script, and nspawn's --resolv-conf=auto
copies (not binds) for a writable host-netns container. systemd-nspawn(1)
states the consequence outright — "no further propagation of configuration is
generally done after the one-time early initialization (this is because the
file is usually updated through copying and renaming)".
dnsmasq has no explicit upstream and follows that file, so a host network
change strands it on a resolver that no longer answers and every non-hive
lookup from every agent hangs. Agents' own resolvers point at the static
bridge IP and never go stale, which is why the symptom presents as "the
gateway needs a kick".
Add a host-side hive-gateway-resolv path unit watching /etc/resolv.conf.
On change it machinectl copy-to's the file into the container and reloads
dnsmasq — ExecReload is kill -HUP, so upstreams are re-read and the cache
flushed without dropping anything; nginx never notices.
- watched from the HOST: a rename on the host doesn't cross the nspawn mount
namespace, so an in-container path unit can't see it (same reason c0re
reloads nginx from the host side)
- copy, not a file bind-mount: openresolv renames over the file, so a bind
would pin the first inode forever — strictly worse than today
- machinectl copy-to writes through the container's own mount namespace, so
this holds regardless of how the container assembles /etc
- armed Before=network-pre.target so the boot's first DHCP write is caught,
and re-run on gateway start for changes made while it was down
- a host file with no nameserver line is skipped, not pushed, so a
mid-rewrite snapshot can't blank hive DNS
- deliberately no fallback server=: dnsmasq queries all known upstreams in
parallel, so a hardcoded public resolver would take a share of normal
traffic rather than only covering the gap
Container nix invocations hard-failed whenever the remote builder
muede-pc2 was unreachable, while the identical build on the host
succeeded. Both go through the same host nix-daemon, so the difference
looked impossible.
The cause is that `fallback` is a client-side option: the nix client
transmits it to the daemon per connection (`tryFallback` in setOptions),
so the caller decides whether a failed remote dispatch may degrade to a
local build — even when the build itself runs on the host daemon under
NIX_REMOTE=daemon. Only genuinely daemon-side settings such as
`builders` are inherited from the host. The containers never set
`fallback`, so they took nix's default of false.
Set it in the agent-container base module and in the CI container, and
correct the hive-ci comment that claimed fallback was inherited from
the host daemon along with buildMachines and max-jobs.
Verified in an agent container: `nix fmt` fails outright on the remote
store's connection reset, while the same command with fallback enabled
reports the same connection error and then builds locally and succeeds.
The host admin socket `/run/hyperhive/host.sock` was `0660 root:root` (no
SocketGroup), so hivectl needed sudo. Group-own it by a new `hive-admin`
group and add a `services.hyperhive.c0re.adminUsers` allowlist: listed users
join `hive-admin` and drive hivectl without root.
- `SocketGroup = "hive-admin"`, `SocketMode = "0660"` on the hive-c0re.socket
unit.
- `/run/hyperhive` -> `0751` (traverse-only, no listing) so the group can reach
the socket path; the socket's own `0660 hive-admin` mode gates the
connection, and the per-agent subdirs keep their own restrictive perms.
- Empty `adminUsers` (the default) leaves `hive-admin` memberless -> root-only,
as before.
The admin socket is full hive control (spawn/kill/destroy/deploy), so
`adminUsers` is an explicit, opt-in trust grant. Documented in
docs/boundary.md (host admin socket access) + docs/tools/hivectl.md.
Implements mouse input by speaking the RFB protocol directly to Weston's
neatvnc server (localhost:HIVE_GUI_VNC_PORT, default 5900) — the VNC
backend's native remote-input path. No /dev/uinput, no kernel bypass;
the compositor mediates all input just as it does for the browser VNC viewer.
Changes:
- rfb_handshake(): RFB 3.8 handshake with security type None (auth-method=none
in weston.ini); shared-session ClientInit keeps the browser viewer connected
- rfb_pointer_event(): encodes a 6-byte RFB PointerEvent (type=5, button-mask,
x/y big-endian)
- rfb_send_pointer_events(): connects, handshakes, sends an event slice,
flushes — all in one TCP connection
- mouse_move(x, y): sends a single PointerEvent(mask=0, x, y)
- mouse_click(x, y, button): sends move → button-down → button-up sequence
(left/middle/right via RFB button-mask bits 0/1/2)
- vnc_port(): reads HIVE_GUI_VNC_PORT from env, falls back to 5900
No new packages or nix options — HIVE_GUI_VNC_PORT is already set by the
harness when gui.enable = true; grim/wtype are the only runtime deps.
Closes#2618.
- run_cmd now returns Result<String, String> — callers pattern-match
instead of comparing against an "ok" sentinel string
- Add cmd_result() helper to format run_cmd results as tool strings
- mouse_click: collapse nested if-let into let-chain (clippy collapsible_if)
- nix fmt: reformat screen.nix package list
New crate hive-screen-mcp: a stdio MCP bridge activated automatically
when an agent has hyperhive.gui.enable = true. Provides five tools:
- screenshot — grim → saves PNG, returns path for Read tool
- type_text — wtype → Unicode text input (no daemon)
- key_press — ydotool key → combos like ctrl+c, super+l
- mouse_move — ydotool mousemove --absolute
- mouse_click — ydotool click, optionally with prior move
New nix/agent-modules/screen.nix: wires the MCP bridge into
extraMcpServers.screen; adds grim + wtype to systemPackages. Adds
hyperhive.gui.screenInput option (default false) which enables the
ydotoold daemon + ydotool for mouse/keyboard injection via /dev/uinput.
screenshot and type_text work without screenInput. key_press,
mouse_move, and mouse_click return a ydotool error until ydotoold is
running and /dev/uinput is accessible in the container.
CI's nix flake check ran in hive-ci's OWN in-container nix-daemon, which
offloads to the pc2 remote builder and HARD-FAILS when pc2 is
unreachable (Connection-reset) — reddening every PR's queue hive-wide.
The host daemon builds fine in the same situation (buildMachines +
max-jobs>=1 + fallback → local build when pc2 is down), and the agent
containers already route through it.
Give hive-ci the same wiring: bind-mount the host nix-daemon socket dir
into the container, set NIX_REMOTE=daemon, and disable the container's
own nix-daemon service + socket. Now CI builds through the host daemon —
pc2 offload when it's up, graceful local fallback when it's down. Drops
the now-moot in-container wait-nix-daemon precond. Needs an operator
rebuild to apply.
The clippy check's comment described `-D warnings -A clippy::pedantic` — the
`-A` half dropping pedantic from the CI gate — but the args were only
`-D warnings`, so pedantic was hard-denied contrary to the doc. Operator
call: pedantic should be gated. Encode that as the single source of truth:
set the workspace lint `pedantic = deny` (errors locally and in CI), and
rewrite the checks.nix comment to match. Args unchanged; `-D warnings` still
gates rustc + non-pedantic clippy warnings. No new failures — the tree was
already pedantic-clean under CI's `-D warnings`, which denied pedantic.
hive-core now fetches each agent's config as a forge-hosted flake input
(git+http://<forge>/agent-configs/<n>.git). Add a git credential helper
(git-credential-hive-forge) that reads the live forge-core admin token on
every invocation and authenticates as the forge core user, wired via the
[credential] stanza in hive-core's $HOME/.gitconfig + on the service PATH.
Reading the token file live means zero stale copies and no resync on
rotation; no token ever lands in a flake URL or lock. Mirrors the existing
github.nix credential-helper pattern.
package-lock.json was updated in 9edd3750 (dompurify/marked/esbuild
bumps) but npmDepsHash in nix/packages/frontend.nix wasn't recomputed,
breaking the frontend FOD build on main (hash mismatch, fixes#2539).
Recomputed via prefetch-npm-deps frontend/package-lock.json.
Completes #2415. Registration no longer gates container@hive-ci start:
- Retire the host-side hive-ci-prefetch.service (+ its 100-line script and
the now-dead coreTokenPath/containerRoot let-bindings) — the forge round-trip
it did on the boot path now lives in hive-c0re (forge/ci_runner.rs), run out
of band during the startup sweep.
- Drop the container@hive-ci TimeoutStartSec = mkForce 180s band-aid that
papered over that boot-path wait tripping the nspawn start timeout (the ~60s
restart loop of #2410).
- gitea-runner-hive gains an ExecStartPre precond (ahead of the nix-daemon
wait) that fails fast unless it is already registered (.runner present) or a
real, non-placeholder token is in place — so missing creds just hold the
runner down instead of blocking the container. Restart=on-failure (no start
limit) self-heals it once hive-c0re writes the token; c0re's explicit restart
is the primary path.
- Set HYPERHIVE_FORGE_CI_ENABLED=1 on hive-c0re.service so the sweep registers.
The tmpfiles TOKEN=placeholder seed + read-only bind-mount are unchanged; the
core admin token still never enters the container.
hive-c0re already samples each agent container's cgroup load for the
dashboard (stats/container_stats.rs); this rides those gauges out to the
configured OTLP endpoint, reusing the existing services.hyperhive.otel
config (endpoint + auth header) — no new toggle.
- New stats/otel_metrics.rs: exports via the OpenTelemetry Rust SDK (same
crates as hive-metric) with the semconv container.* metric names +
container.name attribute so off-the-shelf OTel/Grafana dashboards work,
plus the hive agent label. container.cpu.time (counter, s, from cumulative
cpu.stat usage_usec), container.memory.usage, container.memory.usage.limit;
memory peak / on-disk storage / instantaneous cpu percent stay hyperhive.*
custom (no semconv equivalent). Observable instruments read a shared
snapshot an async task refreshes (gather() is async; SDK callbacks sync).
- container_stats: expose cpu_time_usec (cumulative) on ContainerResource.
- The OTLP auth header is loaded onto hive-c0re's own unit via systemd
LoadCredential and read from $CREDENTIALS_DIRECTORY/otel-headers.
- docs/observability.md documents the host-emitted semconv metrics.
Host-side export, so it covers containers even when their agent is idle.
hive-ci and hive-forge both bind the runtime-generated hive CA cert
read-only and order their container@ unit after hive-tls-ca.service so
the bind source exists before nspawn sets the mount up — the same
bind-mount + ordering + rationale duplicated verbatim in two modules.
Extract that language-agnostic half into a pure helper,
nix/host-modules/lib/hive-ca-trust.nix, taking a container name and
returning { useSelfSigned, caContainerPath, bindMount, containerOrdering }.
The per-runtime consumption stays at each call site (hive-ci's additive
NODE_EXTRA_CA_CERTS, hive-forge's Go SSL_CERT_FILE concat). hive-ci folds
containerOrdering into its existing mkMerge alongside the TimeoutStartSec
bump.
The helper is a pure function, not a module: host-modules/default.nix is
an explicit aggregator (not a glob) and the docs eval imports that same
aggregator, so the lib/ file is never picked up as a module. A third
outbound-TLS-trusting container no longer means a third copy-paste.
Per mara's feedback on PR #2407 ("better: you can also provide url in
dashboard, same as with matrix, no host config"), drops
services.hyperhive.extraForges and the admin-API mint/revoke flow
entirely. The operator now creates a token on the external forge
themselves and pastes a label + base URL + access token into the
dashboard's FORGES tab, the same shape as the GitHub PAT flow plus the
base-URL field from the matrix extra-account flow. hive-c0re only ever
writes/deletes two local files per account (forge-<label>-token,
forge-<label>.json sidecar for the URL) via hive-priv — no remote
account creation, no admin token, no revoke-on-the-remote-side, no nix
config to enumerate.
- nix/host-modules/hive-forge/default.nix: removed the extraForges
option, its label-format assertion, and the HYPERHIVE_EXTRA_FORGES
env forwarding.
- hive-c0re/src/forge/extra.rs: deleted (REST admin-API provisioning,
no longer needed).
- hive-c0re/src/dashboard/extra_forges.rs: GET /api/extra-forges?
agent= lists an agent's stored forges by scanning its state dir
(mirrors matrix_accounts.rs's filename-scan listing), POST
/api/extra-forge-account (agent/label/base_url/token/
action=add|remove) stores or removes an account.
- hive-sh4re/priv_proto.rs + hive-priv/main.rs: new
WriteAgentExtraForgeAccount/DeleteAgentExtraForgeAccount priv
requests (adds base_url, writes/deletes a JSON sidecar alongside the
token).
- hive-c0re/src/priv_client.rs: matching wrapper functions.
- frontend/packages/dashboard/src/credentials.{html,js}: FORGES tab is
a per-agent list + add-account paste form (label/base_url/token), no
grant/revoke-from-catalog UI.
- docs/web-ui/dashboard.md: FORGES tab section rewritten.
Supersedes the design in PR #2407 (already approved+green on the old
admin-API model) — opening as a fresh PR against the same issues
rather than force-pushing over the approved one.
gitea-runner registration (hive-ci-prefetch, host-side) sits on the
boot-critical path -- nspawn readiness is gated on the runner
registering, itself a forge + core-token round trip that can wait up
to 60s for the core token. The default ~60s TimeoutStartSec can trip
mid-register (especially right after a .runner purge, since every
boot re-registers from scratch), killing the half-started container
and triggering a restart loop until the token/forge settle. Bump to
180s so one register attempt has room to finish.
