- mouse_click: error on unrecognised button name instead of silently
treating it as left click; "left" now explicit in match arm
- rfb_handshake: cap ServerInit name-length at 256 bytes to prevent
a multi-GB allocation from an aberrant server response
The display size is not always 1280x720 — it changes when the operator
clicks 'match size' in the web UI. Replace the fixed mention with a
recommendation to use screenshot first to check the current resolution.
Implements mouse input by speaking the RFB protocol directly to Weston's
neatvnc server (localhost:HIVE_GUI_VNC_PORT, default 5900) — the VNC
backend's native remote-input path. No /dev/uinput, no kernel bypass;
the compositor mediates all input just as it does for the browser VNC viewer.
Changes:
- rfb_handshake(): RFB 3.8 handshake with security type None (auth-method=none
in weston.ini); shared-session ClientInit keeps the browser viewer connected
- rfb_pointer_event(): encodes a 6-byte RFB PointerEvent (type=5, button-mask,
x/y big-endian)
- rfb_send_pointer_events(): connects, handshakes, sends an event slice,
flushes — all in one TCP connection
- mouse_move(x, y): sends a single PointerEvent(mask=0, x, y)
- mouse_click(x, y, button): sends move → button-down → button-up sequence
(left/middle/right via RFB button-mask bits 0/1/2)
- vnc_port(): reads HIVE_GUI_VNC_PORT from env, falls back to 5900
No new packages or nix options — HIVE_GUI_VNC_PORT is already set by the
harness when gui.enable = true; grim/wtype are the only runtime deps.
Closes#2618.
- run_cmd now returns Result<String, String> — callers pattern-match
instead of comparing against an "ok" sentinel string
- Add cmd_result() helper to format run_cmd results as tool strings
- mouse_click: collapse nested if-let into let-chain (clippy collapsible_if)
- nix fmt: reformat screen.nix package list
New crate hive-screen-mcp: a stdio MCP bridge activated automatically
when an agent has hyperhive.gui.enable = true. Provides five tools:
- screenshot — grim → saves PNG, returns path for Read tool
- type_text — wtype → Unicode text input (no daemon)
- key_press — ydotool key → combos like ctrl+c, super+l
- mouse_move — ydotool mousemove --absolute
- mouse_click — ydotool click, optionally with prior move
New nix/agent-modules/screen.nix: wires the MCP bridge into
extraMcpServers.screen; adds grim + wtype to systemPackages. Adds
hyperhive.gui.screenInput option (default false) which enables the
ydotoold daemon + ydotool for mouse/keyboard injection via /dev/uinput.
screenshot and type_text work without screenInput. key_press,
mouse_move, and mouse_click return a ydotool error until ydotoold is
running and /dev/uinput is accessible in the container.
