`MATRIX_HTTP` was `http://localhost:8008`, compiled in, used at 18 call
sites. That address is right only while the homeserver happens to share
this daemon's netns, and its doc comment asserted exactly that as a
general fact. A hive whose homeserver lives anywhere else builds fine
and then talks to the wrong machine.
It now reads `HIVE_MATRIX_API_URL`, which `hive-c0re.nix` sets from
`hyperhive.matrix.apiUrl`. The matrix module fills that in with its own
loopback listener when it is the thing running tuwunel — there it is not
a guess but a fact about what it just started — and the operator sets it
by hand otherwise. There is no compiled-in fallback, for the same reason
`forge_http_base()` has none.
`is_present()` follows. It used to scan `nixos-container list` for
`hive-matrix`, which answers "is the homeserver a container on this
host" — a different question, and the reason a remote homeserver would
silently no-op no matter how it was addressed. It now asks whether a URL
is configured. A co-located hive is unaffected: the module supplies the
loopback URL whenever it runs tuwunel itself. It also stops being
`async`, since it no longer does IO, and `require_matrix_present`'s
message names both ways to have a homeserver rather than only the local
container.
Absent a URL, every matrix path no-ops exactly as it did with no
container, and the two accessors make that structural: `Option` for the
callers that fall back to `None`, a `Result` flavour naming the skipped
`is_present()` gate for the ones that propagate.
Third and last of #2860's agent-facing URL fallbacks. The operator's
ruling was "any special casing is done on the nix side - same binaries,
no hard coded fallback", so the default is deleted rather than replaced.
Every layer guessed the same wrong thing, and each guess was only ever
correct for a process sharing the host netns:
- nix/agent-modules/matrix.nix: matrixUrlDefault = localhost:8008, both
as the option's default and as a sentinel the daemon unit compared
against to decide whether to write HIVE_MATRIX_URL. Now nullOr str,
default null, the guard is != null, and the doc says what forge.url's
already says: null means "no matrix", not "guess one".
- nix/host-modules/hive-c0re/environment.nix: forwarded
http://127.0.0.1:<port> when no gatewayHost was set. hive-c0re shares
the host netns so it reads as harmless, but the value is handed to
agents, which do not -- there it names the agent itself. Now forwarded
only when there is a gateway vhost to name, matching the guard
HIVE_MATRIX_PUBLIC_URL already uses twelve lines below.
- hive-matrix-mcp: paths::DEFAULT_HOMESERVER was the same address
compiled in, so dropping the nix defaults alone would have left the
daemon dialling loopback inside the agent's own netns -- the very bug,
one layer down. homeserver_url() is now Option, and an account with no
homeserver is skipped with a log, exactly as one with no token is.
discover_token_accounts already refused to guess for the same reason.
Two comments taught the assumption back to the next reader ("shared host
netns means every agent container resolves localhost to the same
machine"); both now say which side of the netns boundary they describe.
MATRIX_HTTP keeps its value -- hive-c0re really does share the host
netns -- but no longer claims agents do.
Gated with nix eval against the extended agent-base config, as a pair:
with no url set the daemon unit carries no HIVE_MATRIX_URL, and with one
set it carries exactly that. Either check alone passes on a broken guard.
Per mara's standing calibration (#2463): extraction > silencing for
too_many_lines. Splits the space + chat-room provisioning tail into its
own fn, bringing ensure_all back under the 100-line threshold without
an #[allow].
Fast-follow for #2289 — matrix::ensure_all() (admin user, per-agent
sync, hive Space + chat-room provisioning/invites) ran periodically
every 30 minutes but only ever warn!'d to the journal on failure, so a
persistent problem (missing invites, broken admin token, etc.) was
invisible to the operator.
ensure_all() now returns bool (aggregate ok/fail across every
sub-step) instead of (), and both call sites in main.rs feed that into
a debounced SweepHealth("matrix_ensure_all", warn, threshold=2) —
matches the existing knowledge_pull pattern. A lone bad sweep
self-heals silently; two consecutive failures raise a banner that
clears on the next clean sweep.
forge::ensure_all()'s remaining independent steps are still open —
that sweep only runs once at startup (no periodic loop), so the
debounced pattern doesn't map as directly; left for a follow-up.
The restricted join rule needs room version >= 8. Pin it explicitly so a
future homeserver-default change can't silently invalidate the restricted
rule (which would fall back to invite-only and break the operator-joins-
from-the-Space path). Per review on the PR.
The hive Space was created empty — joining it surfaced no rooms because
Matrix doesn't auto-join a Space's children. Provision a default
"hive-chat" room on the matrix sweep, wire it bidirectionally to the
Space (m.space.child on the Space, m.space.parent on the room), and
invite @hive + every agent. The room uses a restricted join rule
allowing any Space member to join, so the operator (a Space member) can
join it from the Space hierarchy without an explicit invite.
Idempotent, mirroring ensure_hive_space: persisted chat-room-id wins,
else rediscover a non-space room named hive-chat, else createRoom. The
space-child link is re-applied each sweep (idempotent PUT) so a
recovered room reconverges its hierarchy link. Room id persisted to
matrix/chat-room-id (0600, survives destroy --purge).
Per mara's review: drop the #hive:<server> room alias (special chars) and
rediscover the canonical Space by its hardcoded plain name instead.
ensure_hive_space dedup is now:
1. room-id file present -> reuse it
2. else scan the admin's joined rooms for the m.space named HIVE_SPACE_NAME
('hive') and adopt the first match (re-persisting the file) -> recovers
the existing space after a state wipe instead of creating a duplicate
3. else createRoom (plain name, no alias)
find_space_by_name walks /joined_rooms and checks each room's m.room.create
type == m.space and m.room.name == 'hive'. No alias, no special-char anchor.
server_name is no longer needed by ensure_hive_space (dropped the param).
ensure_hive_space relied solely on the persisted room-id file. If that file
is ever lost (a full /var/lib/hyperhive wipe), the next sweep blind-creates a
new m.space — the homeserver keeps the old one, so duplicate hive spaces
accumulate (observed: multiple 'hive'/'pr1ma' rooms on the live instance).
Anchor the Space to a stable canonical alias #hive:<server>:
- fast path (room-id file present): reuse it and heal the alias mapping so
it keeps pointing at the canonical room
- no file: resolve #hive:<server> and adopt the existing room if present,
re-persisting the file — recovers the space after a wipe instead of
duplicating it
- only create (with the alias) when neither yields a room
server_name is now discovered before ensure_hive_space in ensure_all and
threaded through (the alias needs it). Existing deployments heal the alias
onto their current space on the next sweep; no new room is created when the
file is present.
Full `nix flake check` (CI) runs the treefmt formatting derivation. While the
hive-ci runner was offline (#1221), PRs merged without it, leaving 5 files
unformatted: hive-ag3nt/src/web_ui.rs, hive-c0re/src/bin/hivectl.rs,
hive-c0re/src/knowledge.rs, hive-c0re/src/matrix.rs,
hive-forge/src/verbs/attachment_get.rs. `nix fmt` output, pure formatting.
Per operator review on the PR: the prose marker list ('new password is:',
' to:', 'changed to:', etc.) was speculative — built from a misread
screenshot, not a real reply. The conduit admin bot always code-spans the
password, so the backtick anchor is the verified, complete format. Removing
the markers leaves a ~15-line function that's honest about what it parses.
If a non-code-span format ever appears, extract_new_password returns None and
the diagnostic logging in admin_room_send_and_poll records the raw body, so a
real format change is visible — far better than a speculative marker silently
mis-parsing it. Tests now cover the live format, symbol passwords, the
code-spanned-user-id error guard, and the no-codespan / non-password cases.
The live conduit admin-room reply (observed directly in #admins) is:
Successfully reset the password for user @x:server: `<password>`
The delimiter is ': ' after the user id, NOT ' to:' — so the ' to: ' marker
added earlier never matched the real format, and extract_new_password returned
None for every reply, timing out auto-recovery for every agent.
conduit always renders the password as a backtick code span, so anchor on
that directly: take the content of the first backtick pair when the message
is a password-reset success. Guard against a code-spanned matrix user id in
an error message (a real password has no whitespace and isn't @localpart:server).
Prose markers stay as a fallback for hypothetical non-code-span builds.
Tests cover the exact live format and the code-spanned-user-id error case.
mara observed the tuwunel reply renders the new password as a code span,
so the plain message body carries literal backticks:
Successfully reset password for @user:server to: `<password>`
The previous extraction stopped at the first whitespace, capturing the
surrounding backticks ("`<password>`") and producing a login string that
doesn't match the password the bot actually set — recovery would still
fail after parsing.
Strip a leading backtick after the marker and stop the token at the first
whitespace OR closing backtick. Generated passwords contain neither, so a
real password is never truncated mid-token. Two regression tests cover the
backtick-wrapped form, including trailing prose after the closing backtick.
extract_new_password had markers for 'is:', 'changed to:', 'reset to:',
'set to:' etc. but none match tuwunel's actual reset-password reply:
Successfully reset password for @user:server to: <password>
Here the verb ('reset') is not adjacent to 'to:', so every marker missed
and the function returned None for every polled event. The admin-room poll
then ran its full 15s loop without a match and the auto-recovery timed out
on every agent — even though the bot replied correctly and the backward
poll found the message. This is why matrix password auto-recovery kept
looping despite the poll-strategy fix.
Add a generic ' to: ' marker (with surrounding spaces) placed after the
specific verb markers and before the bare 'password:' last resort. Matrix
user ids and server names can't contain ' to: ', so it only ever anchors
on the prose delimiter. Two regression tests cover the exact production
wording.
The forward-pagination approach (dir=b anchor → dir=f poll) fails in
production: commands sent as @hive time out consistently even though
tuwunel responds. Root cause is likely a pagination-token direction
incompatibility in some tuwunel builds where 'end' from dir=b cannot
be used as 'from' for dir=f.
New strategy: send the command, capture the event_id from the PUT
response, then poll dir=b&limit=20 each tick. Events come back
newest-first; walk until we hit our own event_id, then stop — anything
before that marker arrived after our command. Simpler, avoids stored
tokens entirely.
Also:
- check formatted_body in addition to body (some admin bots put content
only in HTML formatted_body)
- add more extract_new_password patterns: 'changed to:', 'reset to:',
'set to:', 'new password:', 'password:' to handle different tuwunel
version response formats
- add unit tests for new patterns
Fixes#1283.
tuwunel can take longer than 5 seconds to process admin-room commands
during startup when the homeserver is under load. Bump both the poll
count (5→15) and the timeout message strings to match.
tuwunel can take longer than 5 seconds to process admin-room commands
during startup when the homeserver is under load. Bump poll count 5→15
and update timeout message strings to match.
tuwunel 1.6.x does not implement the Synapse admin REST API.
reset_user_password() now falls back to the Matrix admin room
(#admins:<server>) when PUT /_synapse/admin/v2/users returns 404:
1. Discover admin room ID via #admins:<server> alias
2. Get current messages end-token (pagination anchor)
3. Send 'reset-password @<localpart>:<server>' as @hive admin user
4. Poll for bot response up to 5 x 1s; extract password from message
5. Persist the new password and return it
The function signature changes from Result<()> to Result<String> so the
caller can use the effective password (which may be server-generated on
the admin-room path) for subsequent login calls.
Closes#1267.
Addresses mara's review: each credential type gets its own PrivRequest
variant, making the exact priv surface visible in the wire protocol.
No runtime filename dispatch — the operation name is the gate.
- WriteAgentForgeToken { agent_name, token } → state/forge-token
- WriteAgentMatrixToken { agent_name, token } → state/matrix-token
- priv_client: two typed fns (write_agent_forge_token, write_agent_matrix_token)
- forge.rs: split mint_and_persist_token into mint_and_persist_agent_token
(priv) + mint_and_persist_core_token (direct write); drop dead token_path fn
- matrix.rs: call write_agent_matrix_token directly
hive-c0re runs as the unprivileged hive-core user (privsep from #702)
and cannot write to agent-owned state directories. forge-token and
matrix-token writes were failing with EACCES on every startup sweep.
Add WriteAgentStateFile to PrivRequest: hive-priv (root) writes the
file 0600 and chowns it to the agent user so the agent can read it.
- hive-sh4re: add AGENT_STATE_ROOT constant + WriteAgentStateFile variant
- hive-priv: validate agent name + filename (no traversal), write via root
- priv_client: add write_agent_state_file helper
- forge: mint_and_persist_token routes agent paths through priv
- matrix: ensure_user_for routes matrix-token through priv
Closes#1257
forge_admin() spawned nixos-container run hive-forge directly from the
hive-core process. nixos-container run uses nsenter to enter the container
namespaces, which requires root. hive-core is unprivileged, so every call
failed with: nsenter: stat of /proc/<pid>/ns/user failed: Permission denied
Fix: add RunForgeAdmin { args } to PrivRequest. hive-priv (root) handles
it by spawning nixos-container run hive-forge -- runuser -u forgejo --
forgejo --work-path /var/lib/forgejo admin <args>. forge_admin() now calls
priv_client::run_forge_admin().
matrix: ensure_user_for hit M_USER_IN_USE then failed when the stored
password file was missing (state dirs wiped but homeserver kept accounts).
Previously required manual hivectl matrix reset-password <name>.
Fix: add auto_reset_password() — calls the admin API (PUT
/_synapse/admin/v2/users/@<name>:<server> with the hive admin token) to
set a new random password, then proceeds with login. Falls back to the
existing manual-recovery error if the admin token is unavailable.
Closes#1234
- drop stale --server flag reference from ensure_admin_user doc comment
- simplify M_USER_IN_USE recovery message: point at hivectl commands
- add #[must_use] to admin_token_path()
- rename tracing field rename_error in migration warn log (was error,
which held rename err but fired on read failure — misleading)
Add matrix::discover_server_name() via GET /_matrix/key/v2/server
(unauthenticated federation endpoint, always returns server_name).
hivectl is always talking to the local hive — no reason to require
the operator to spell out the server_name.
- provision @hive:<server> as the first matrix account in ensure_all()
(Conduit/tuwunel makes the first registered user admin automatically)
- add hivectl matrix sync-admin — manual re-provision of the admin token
- add hivectl matrix promote-user <name> --server <name> — promote via
Synapse-compat admin API using the hive admin token
- add hivectl matrix reset-password <name> --server <name> — reset an
agent's password + persist it so ensure_user_for can re-login; follow
with hivectl matrix create-user <name> to mint a fresh access token
- both commands fall back to HYPERHIVE_MATRIX_SERVER_NAME env var for
--server when omitted
The password file was stored at agent_notes_dir/matrix-password which
lives inside agent_state_root — wiped by destroy --purge. On re-spawn
with the same agent name, the matrix user still exists in the homeserver
but the stored password is gone, making re-login impossible.
Move password to /var/lib/hyperhive/matrix-creds/<name>-password which
is not deleted by purge. On re-spawn, ensure_user_for finds M_USER_IN_USE,
reads the preserved password, re-logins, and writes a fresh token.
Also:
- add one-time migration that moves existing passwords from the old path
to the new location on first access after upgrade
- remove chown_to_agent on the password file (it is now host-only, not
inside the agent bind-mount tree)
- fix the error message to give actionable recovery steps instead of
suggesting hivectl matrix create-user --password which is rejected for
agent accounts
hive-priv had no PATH, so the nixos-container it runs for every container
op (incl. lifecycle::list) failed ENOENT. build_all swallows that into an
empty list — the 'no managed containers' symptom. Give the helper the
minimal set nixos-container shells out to (nixos-container, nix, util-linux,
e2fsprogs) on top of the systemd/coreutils/findutils already in the unit
PATH, rather than the whole system profile.
Also route forge/matrix is_present() through priv_client::list_containers
instead of spawning nixos-container directly from unprivileged hive-c0re.
When the token file is deleted but the homeserver account still exists,
register_user returns M_USER_IN_USE (HTTP 400) and the provisioning
sweep hard-fails, leaving the agent without a working matrix token.
Fix: persist the random password to matrix-password alongside the
access token on first registration. On subsequent attempts where
M_USER_IN_USE is returned, fall back to login_user (m.login.password)
using the stored password. If both files are gone, the error message
guides the operator to `hivectl matrix create-user <name> --password`.
