Per mara's standing calibration (#2463): extraction > silencing for
too_many_lines. Splits the space + chat-room provisioning tail into its
own fn, bringing ensure_all back under the 100-line threshold without
an #[allow].
Fast-follow for #2289 — matrix::ensure_all() (admin user, per-agent
sync, hive Space + chat-room provisioning/invites) ran periodically
every 30 minutes but only ever warn!'d to the journal on failure, so a
persistent problem (missing invites, broken admin token, etc.) was
invisible to the operator.
ensure_all() now returns bool (aggregate ok/fail across every
sub-step) instead of (), and both call sites in main.rs feed that into
a debounced SweepHealth("matrix_ensure_all", warn, threshold=2) —
matches the existing knowledge_pull pattern. A lone bad sweep
self-heals silently; two consecutive failures raise a banner that
clears on the next clean sweep.
forge::ensure_all()'s remaining independent steps are still open —
that sweep only runs once at startup (no periodic loop), so the
debounced pattern doesn't map as directly; left for a follow-up.
The restricted join rule needs room version >= 8. Pin it explicitly so a
future homeserver-default change can't silently invalidate the restricted
rule (which would fall back to invite-only and break the operator-joins-
from-the-Space path). Per review on the PR.
The hive Space was created empty — joining it surfaced no rooms because
Matrix doesn't auto-join a Space's children. Provision a default
"hive-chat" room on the matrix sweep, wire it bidirectionally to the
Space (m.space.child on the Space, m.space.parent on the room), and
invite @hive + every agent. The room uses a restricted join rule
allowing any Space member to join, so the operator (a Space member) can
join it from the Space hierarchy without an explicit invite.
Idempotent, mirroring ensure_hive_space: persisted chat-room-id wins,
else rediscover a non-space room named hive-chat, else createRoom. The
space-child link is re-applied each sweep (idempotent PUT) so a
recovered room reconverges its hierarchy link. Room id persisted to
matrix/chat-room-id (0600, survives destroy --purge).
Per mara's review: drop the #hive:<server> room alias (special chars) and
rediscover the canonical Space by its hardcoded plain name instead.
ensure_hive_space dedup is now:
1. room-id file present -> reuse it
2. else scan the admin's joined rooms for the m.space named HIVE_SPACE_NAME
('hive') and adopt the first match (re-persisting the file) -> recovers
the existing space after a state wipe instead of creating a duplicate
3. else createRoom (plain name, no alias)
find_space_by_name walks /joined_rooms and checks each room's m.room.create
type == m.space and m.room.name == 'hive'. No alias, no special-char anchor.
server_name is no longer needed by ensure_hive_space (dropped the param).
ensure_hive_space relied solely on the persisted room-id file. If that file
is ever lost (a full /var/lib/hyperhive wipe), the next sweep blind-creates a
new m.space — the homeserver keeps the old one, so duplicate hive spaces
accumulate (observed: multiple 'hive'/'pr1ma' rooms on the live instance).
Anchor the Space to a stable canonical alias #hive:<server>:
- fast path (room-id file present): reuse it and heal the alias mapping so
it keeps pointing at the canonical room
- no file: resolve #hive:<server> and adopt the existing room if present,
re-persisting the file — recovers the space after a wipe instead of
duplicating it
- only create (with the alias) when neither yields a room
server_name is now discovered before ensure_hive_space in ensure_all and
threaded through (the alias needs it). Existing deployments heal the alias
onto their current space on the next sweep; no new room is created when the
file is present.
Full `nix flake check` (CI) runs the treefmt formatting derivation. While the
hive-ci runner was offline (#1221), PRs merged without it, leaving 5 files
unformatted: hive-ag3nt/src/web_ui.rs, hive-c0re/src/bin/hivectl.rs,
hive-c0re/src/knowledge.rs, hive-c0re/src/matrix.rs,
hive-forge/src/verbs/attachment_get.rs. `nix fmt` output, pure formatting.
Per operator review on the PR: the prose marker list ('new password is:',
' to:', 'changed to:', etc.) was speculative — built from a misread
screenshot, not a real reply. The conduit admin bot always code-spans the
password, so the backtick anchor is the verified, complete format. Removing
the markers leaves a ~15-line function that's honest about what it parses.
If a non-code-span format ever appears, extract_new_password returns None and
the diagnostic logging in admin_room_send_and_poll records the raw body, so a
real format change is visible — far better than a speculative marker silently
mis-parsing it. Tests now cover the live format, symbol passwords, the
code-spanned-user-id error guard, and the no-codespan / non-password cases.
The live conduit admin-room reply (observed directly in #admins) is:
Successfully reset the password for user @x:server: `<password>`
The delimiter is ': ' after the user id, NOT ' to:' — so the ' to: ' marker
added earlier never matched the real format, and extract_new_password returned
None for every reply, timing out auto-recovery for every agent.
conduit always renders the password as a backtick code span, so anchor on
that directly: take the content of the first backtick pair when the message
is a password-reset success. Guard against a code-spanned matrix user id in
an error message (a real password has no whitespace and isn't @localpart:server).
Prose markers stay as a fallback for hypothetical non-code-span builds.
Tests cover the exact live format and the code-spanned-user-id error case.
mara observed the tuwunel reply renders the new password as a code span,
so the plain message body carries literal backticks:
Successfully reset password for @user:server to: `<password>`
The previous extraction stopped at the first whitespace, capturing the
surrounding backticks ("`<password>`") and producing a login string that
doesn't match the password the bot actually set — recovery would still
fail after parsing.
Strip a leading backtick after the marker and stop the token at the first
whitespace OR closing backtick. Generated passwords contain neither, so a
real password is never truncated mid-token. Two regression tests cover the
backtick-wrapped form, including trailing prose after the closing backtick.
extract_new_password had markers for 'is:', 'changed to:', 'reset to:',
'set to:' etc. but none match tuwunel's actual reset-password reply:
Successfully reset password for @user:server to: <password>
Here the verb ('reset') is not adjacent to 'to:', so every marker missed
and the function returned None for every polled event. The admin-room poll
then ran its full 15s loop without a match and the auto-recovery timed out
on every agent — even though the bot replied correctly and the backward
poll found the message. This is why matrix password auto-recovery kept
looping despite the poll-strategy fix.
Add a generic ' to: ' marker (with surrounding spaces) placed after the
specific verb markers and before the bare 'password:' last resort. Matrix
user ids and server names can't contain ' to: ', so it only ever anchors
on the prose delimiter. Two regression tests cover the exact production
wording.
The forward-pagination approach (dir=b anchor → dir=f poll) fails in
production: commands sent as @hive time out consistently even though
tuwunel responds. Root cause is likely a pagination-token direction
incompatibility in some tuwunel builds where 'end' from dir=b cannot
be used as 'from' for dir=f.
New strategy: send the command, capture the event_id from the PUT
response, then poll dir=b&limit=20 each tick. Events come back
newest-first; walk until we hit our own event_id, then stop — anything
before that marker arrived after our command. Simpler, avoids stored
tokens entirely.
Also:
- check formatted_body in addition to body (some admin bots put content
only in HTML formatted_body)
- add more extract_new_password patterns: 'changed to:', 'reset to:',
'set to:', 'new password:', 'password:' to handle different tuwunel
version response formats
- add unit tests for new patterns
Fixes#1283.
tuwunel can take longer than 5 seconds to process admin-room commands
during startup when the homeserver is under load. Bump both the poll
count (5→15) and the timeout message strings to match.
tuwunel can take longer than 5 seconds to process admin-room commands
during startup when the homeserver is under load. Bump poll count 5→15
and update timeout message strings to match.
tuwunel 1.6.x does not implement the Synapse admin REST API.
reset_user_password() now falls back to the Matrix admin room
(#admins:<server>) when PUT /_synapse/admin/v2/users returns 404:
1. Discover admin room ID via #admins:<server> alias
2. Get current messages end-token (pagination anchor)
3. Send 'reset-password @<localpart>:<server>' as @hive admin user
4. Poll for bot response up to 5 x 1s; extract password from message
5. Persist the new password and return it
The function signature changes from Result<()> to Result<String> so the
caller can use the effective password (which may be server-generated on
the admin-room path) for subsequent login calls.
Closes#1267.
Addresses mara's review: each credential type gets its own PrivRequest
variant, making the exact priv surface visible in the wire protocol.
No runtime filename dispatch — the operation name is the gate.
- WriteAgentForgeToken { agent_name, token } → state/forge-token
- WriteAgentMatrixToken { agent_name, token } → state/matrix-token
- priv_client: two typed fns (write_agent_forge_token, write_agent_matrix_token)
- forge.rs: split mint_and_persist_token into mint_and_persist_agent_token
(priv) + mint_and_persist_core_token (direct write); drop dead token_path fn
- matrix.rs: call write_agent_matrix_token directly
hive-c0re runs as the unprivileged hive-core user (privsep from #702)
and cannot write to agent-owned state directories. forge-token and
matrix-token writes were failing with EACCES on every startup sweep.
Add WriteAgentStateFile to PrivRequest: hive-priv (root) writes the
file 0600 and chowns it to the agent user so the agent can read it.
- hive-sh4re: add AGENT_STATE_ROOT constant + WriteAgentStateFile variant
- hive-priv: validate agent name + filename (no traversal), write via root
- priv_client: add write_agent_state_file helper
- forge: mint_and_persist_token routes agent paths through priv
- matrix: ensure_user_for routes matrix-token through priv
Closes#1257
forge_admin() spawned nixos-container run hive-forge directly from the
hive-core process. nixos-container run uses nsenter to enter the container
namespaces, which requires root. hive-core is unprivileged, so every call
failed with: nsenter: stat of /proc/<pid>/ns/user failed: Permission denied
Fix: add RunForgeAdmin { args } to PrivRequest. hive-priv (root) handles
it by spawning nixos-container run hive-forge -- runuser -u forgejo --
forgejo --work-path /var/lib/forgejo admin <args>. forge_admin() now calls
priv_client::run_forge_admin().
matrix: ensure_user_for hit M_USER_IN_USE then failed when the stored
password file was missing (state dirs wiped but homeserver kept accounts).
Previously required manual hivectl matrix reset-password <name>.
Fix: add auto_reset_password() — calls the admin API (PUT
/_synapse/admin/v2/users/@<name>:<server> with the hive admin token) to
set a new random password, then proceeds with login. Falls back to the
existing manual-recovery error if the admin token is unavailable.
Closes#1234
- drop stale --server flag reference from ensure_admin_user doc comment
- simplify M_USER_IN_USE recovery message: point at hivectl commands
- add #[must_use] to admin_token_path()
- rename tracing field rename_error in migration warn log (was error,
which held rename err but fired on read failure — misleading)
Add matrix::discover_server_name() via GET /_matrix/key/v2/server
(unauthenticated federation endpoint, always returns server_name).
hivectl is always talking to the local hive — no reason to require
the operator to spell out the server_name.
- provision @hive:<server> as the first matrix account in ensure_all()
(Conduit/tuwunel makes the first registered user admin automatically)
- add hivectl matrix sync-admin — manual re-provision of the admin token
- add hivectl matrix promote-user <name> --server <name> — promote via
Synapse-compat admin API using the hive admin token
- add hivectl matrix reset-password <name> --server <name> — reset an
agent's password + persist it so ensure_user_for can re-login; follow
with hivectl matrix create-user <name> to mint a fresh access token
- both commands fall back to HYPERHIVE_MATRIX_SERVER_NAME env var for
--server when omitted
The password file was stored at agent_notes_dir/matrix-password which
lives inside agent_state_root — wiped by destroy --purge. On re-spawn
with the same agent name, the matrix user still exists in the homeserver
but the stored password is gone, making re-login impossible.
Move password to /var/lib/hyperhive/matrix-creds/<name>-password which
is not deleted by purge. On re-spawn, ensure_user_for finds M_USER_IN_USE,
reads the preserved password, re-logins, and writes a fresh token.
Also:
- add one-time migration that moves existing passwords from the old path
to the new location on first access after upgrade
- remove chown_to_agent on the password file (it is now host-only, not
inside the agent bind-mount tree)
- fix the error message to give actionable recovery steps instead of
suggesting hivectl matrix create-user --password which is rejected for
agent accounts
hive-priv had no PATH, so the nixos-container it runs for every container
op (incl. lifecycle::list) failed ENOENT. build_all swallows that into an
empty list — the 'no managed containers' symptom. Give the helper the
minimal set nixos-container shells out to (nixos-container, nix, util-linux,
e2fsprogs) on top of the systemd/coreutils/findutils already in the unit
PATH, rather than the whole system profile.
Also route forge/matrix is_present() through priv_client::list_containers
instead of spawning nixos-container directly from unprivileged hive-c0re.
When the token file is deleted but the homeserver account still exists,
register_user returns M_USER_IN_USE (HTTP 400) and the provisioning
sweep hard-fails, leaving the agent without a working matrix token.
Fix: persist the random password to matrix-password alongside the
access token on first registration. On subsequent attempts where
M_USER_IN_USE is returned, fall back to login_user (m.login.password)
using the stored password. If both files are gone, the error message
guides the operator to `hivectl matrix create-user <name> --password`.
