vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(#2593): mark forge notifications read on broker-delivery

Browse source
This commit is contained in:
damocles 2026-07-19 18:21:04 +02:00 • committed by mara
commit da056d0043
3 changed files with 104 additions and 212 deletions
Download patch file Download diff file Expand all files Collapse all files

92
docs/forge.md
View file

@ -91,66 +91,48 @@ unread-only filter), formats each notification as a broker
`Wake { from: "forge" }` message, and delivers it to the agent's own `Wake { from: "forge" }` message, and delivers it to the agent's own
inbox so claude's normal turn loop picks it up. inbox so claude's normal turn loop picks it up.
### Mark-read on read, not on delivery ### Mark-read on delivery
Delivered conversation threads are deliberately left **unread** in On a **successful** broker delivery, `forge_notify` marks the thread
forge. The hive-forge read-before-comment guard keys off forge's own read on forge straight away (`PATCH /notifications/threads/{id}`). The
notification read-state (`GET /notifications?all=false`) to refuse a broker inbox is the durable work queue now — each delivered wake is a
comment when a thread has unread activity by others — so the agent sqlite row with its own ack lifecycle — so the forge unread flag no
reading the thread via the CLI (`hive-forge comments` / `view`, longer needs to track whether the agent has *processed* a
which `PATCH`es `/notifications/threads/{id}`) is the single notification. Clearing it on delivery keeps forge's unread set **tiny
mark-read point. If `forge_notify` marked threads read on delivery, by construction**: at rest it holds only threads that failed to
that unread signal would be consumed before the agent acts and the deliver plus whatever arrived since the last 30s poll.
guard could never fire.
Because a delivered thread stays unread, it reappears in every That size property is the whole point. A container rebuild starts the
`?all=false` poll. A **delivery-dedupe cursor** (thread id → poller with no memory of what it delivered, re-scans `?all=false`, and
last-delivered `updated_at`) stops the same version from re-firing a finds nothing stale — the delivered threads are already read on forge.
wake; a new comment bumps `updated_at` so genuinely new activity Forge's own read-state is thus the durable, cross-rebuild record of
re-delivers. The cursor is pure anti-spam, not a correctness oracle. what's been delivered; there is **no persisted cursor**. (This
Each poll prunes it to the threads still in the unread set. A failed replaced an earlier design that left threads unread and leaned on a
wake delivery is left unread **and** out of the cursor, so it persisted dedup cursor: a rebuild that lost the cursor re-delivered the
resurfaces next tick. entire still-unread backlog as fresh wakes — the notification flood of
#2593 / #2106.)
**Size bound:** the per-poll prune retains only ids present in the **Read-before-comment coupling, dropped on purpose.** The old design
single `limit=UNREAD_FETCH_LIMIT` (50) fetch page, so the cursor never left threads unread so the hive-forge read-before-comment guard (which
exceeds that many entries — it tracks the unread _window_, not the keys off forge unread-state) would force the agent to view a thread
all-time notification count. The fetch limit and the bound are the before commenting. That coupling is gone: the broker wake already
same constant in `forge_notify.rs` (with a debug assertion), so a carries the notification body, so *delivery is the read*. An agent that
future pagination change grows the ceiling visibly rather than wants the full thread still runs `hive-forge comments` / `view`; the
silently. This is why the cursor stays a small JSON field rather than guard no longer blocks a first comment on a freshly-delivered thread.
a db table — see the storage discussion on the tracker (issue 2117).
The cursor is **persisted** as the `forge_cursor` field of the **In-process dedupe (tiny, ephemeral).** A single-process map (thread
harness's consolidated `hyperhive-harness.json` state file (atomic id → last-delivered `updated_at`) guards the narrow window where a
tmp+rename, flushed only when it changed) and reloaded on boot, so a mark-read call *transiently fails* and the thread reappears unread in
container rebuild/restart doesn't re-deliver the whole currently-unread the next poll before its `updated_at` bumps — so a flaky PATCH doesn't
backlog (#2106 — previously the in-memory-only cursor was lost on re-fire the wake. It is **not persisted** and resets on restart (forge
restart and every old still-unread thread re-fired a wake). The poller read-state covers the durable case). Each poll prunes it to the ids in
runs in the same harness process that owns that file, so it's one the single `limit=UNREAD_FETCH_LIMIT` (50) fetch page, so it can never
daemon → one state file rather than a second json; both writers exceed that many entries (a debug assertion pins the invariant; the
(turn-loop fields + this cursor) go read-modify-write under a shared fetch limit and the bound are the same constant). A failed *delivery*
lock so neither clobbers the other's fields. This is safe because a is left unread and out of the map, so it resurfaces next tick.
thread is recorded **after** a successful broker delivery, and the
broker inbox is durable sqlite — so a persisted "delivered" entry can
never swallow a wake the agent never received. Crucially the cursor is
a private dedup mirror, **not** forge's read-state: it does not
reintroduce the read-before-comment coupling that ruled out the old
mark-read-on-delivery approach. A missing (first boot) or malformed
cursor degrades to empty — re-deliver the unread set once — never an
abort.
Self-echo notifications (the agent's own writes, see below) are the Self-echo notifications (the agent's own writes, see below) are marked
one path still marked-read directly (no read-before-comment value). read directly without a delivery — same `mark_read` call, no wake.
> Note: the unread list grows for threads the agent never reads via
> the CLI, since nothing else trims it. This does not affect guard
> correctness (the guard does a per-thread, repo-scoped query) nor
> wake delivery (Forgejo orders unread newest-first, so new activity
> always lands in the polled window). Bounding the unread list is a
> separate follow-up — explicit subscription management via a
> hive-forge CLI verb, rather than the poller second-guessing which
> repo watches to drop.
### Activation gates (graceful no-ops) ### Activation gates (graceful no-ops)

148
hive-agent/src/forge_notify.rs
View file

@ -1,18 +1,19 @@
//! Background Forgejo notification poller. Polls //! Background Forgejo notification poller. Polls
//! `GET /notifications?all=false` every 30s, formats each unread //! `GET /notifications?all=false` every 30s, formats each unread
//! notification as a broker `Wake { from: "forge" }` message, and //! notification as a broker `Wake { from: "forge" }` message, delivers it
//! delivers it to the agent's inbox. Delivered threads are deliberately //! to the agent's inbox, and — on a successful delivery — marks the thread
//! left UNREAD in forge — the hive-forge read-before-comment guard keys //! read on forge straight away. The broker inbox is the durable work queue
//! off forge's own unread-state, and the agent reading the thread via the //! (each row has its own ack lifecycle), so the forge unread flag no longer
//! CLI is what marks it read. A delivery-dedupe cursor (thread id → //! needs to track agent processing: clearing it on delivery keeps forge's
//! last-delivered `updated_at`) stops the still-unread notification from //! unread set tiny by construction, so a container rebuild's re-scan of
//! re-firing a wake every poll; self-echo notifications (the agent's own //! `?all=false` finds nothing stale and cannot re-deliver a backlog. Forge's
//! writes) are still marked read directly. The cursor is persisted as the //! own read-state is thus the durable, cross-rebuild record of what's been
//! `forge_cursor` field of the harness's consolidated `hyperhive-harness.json` //! delivered — there is no persisted cursor. A small in-process dedupe map
//! (via [`crate::events`]) and reloaded on boot so a container //! (thread id → last-delivered `updated_at`) only guards the narrow window
//! rebuild/restart doesn't re-deliver the whole currently-unread backlog — //! where a mark-read call transiently fails and the thread reappears unread
//! it is a private dedup mirror, NOT forge's read-state, so the //! before its `updated_at` bumps; it is ephemeral and reset on restart.
//! read-before-comment guard is untouched. //! Self-echo notifications (the agent's own writes) are marked read without
//! a delivery.
//! //!
//! Activation gates, self-notification filtering, body excerpt + //! Activation gates, self-notification filtering, body excerpt +
//! truncation + heading escape, wrapper formats (comment / review / //! truncation + heading escape, wrapper formats (comment / review /
@ -37,10 +38,9 @@ const POLL_INTERVAL_SECS: u64 = 30;
/// `forgejo-api` client (which exposes no timeout knob of its own). /// `forgejo-api` client (which exposes no timeout knob of its own).
const HTTP_TIMEOUT_SECS: u64 = 10; const HTTP_TIMEOUT_SECS: u64 = 10;
/// Page size of the unread-notifications fetch. This is also the hard /// Page size of the unread-notifications fetch. This is also the hard
/// bound on the persisted delivery-dedupe cursor: each poll prunes the /// bound on the in-process dedupe map: each poll prunes the map to the
/// cursor to the ids in this window, so the `forge_cursor` field in /// ids in this window, so it can never exceed this many entries. Keep
/// `hyperhive-harness.json` can never exceed this many entries. Keep /// the two coupled — bumping the fetch limit grows the map's ceiling
/// the two coupled — bumping the fetch limit grows the cursor's ceiling
/// with it, deliberately and visibly. /// with it, deliberately and visibly.
const UNREAD_FETCH_LIMIT: usize = 50; const UNREAD_FETCH_LIMIT: usize = 50;
/// Maximum characters of a body/comment to include in the wake message. /// Maximum characters of a body/comment to include in the wake message.
@ -160,31 +160,17 @@ pub async fn run(socket: PathBuf) {
info!(forge_url = %forge_url, "forge_notify: polling started"); info!(forge_url = %forge_url, "forge_notify: polling started");
// Delivery-dedupe cursor: notification thread id -> the `updated_at` // In-process delivery-dedupe map: notification thread id -> the
// of the version we last woke the agent for. We no longer mark a // `updated_at` of the version we last woke the agent for. A delivered
// thread read on delivery (that would consume the unread signal the // thread is marked read on forge (in `poll_once`), so it drops out of
// hive-forge read-before-comment guard relies on), so this map is what // `?all=false` next poll and never re-fires; this map only guards the
// stops the same unread notification from re-firing a wake every poll. // narrow window where a mark-read call transiently fails and the thread
// A new comment bumps `updated_at`, so the thread re-delivers. This is // reappears unread before its `updated_at` bumps. It is deliberately
// purely anti-spam, NOT a correctness oracle. // ephemeral (NOT persisted): forge's own read-state is the durable,
// // cross-rebuild source of truth for what's been delivered, so a rebuild
// It is persisted as the `forge_cursor` field of the harness's // starts with an empty map and re-scans only the genuinely-still-unread
// consolidated state file and reloaded here on boot so a container // set — which is tiny by construction because delivery marks read.
// rebuild/restart doesn't re-deliver the entire currently-unread let mut delivered: HashMap<u64, String> = HashMap::new();
// backlog. Persisting is safe because we only record a thread AFTER a
// successful broker delivery, and the broker inbox is durable sqlite —
// so a persisted "delivered" entry can never swallow a wake the agent
// never received. The cursor is a private dedup mirror, decoupled from
// forge's own read-state, so it doesn't reintroduce the
// read-before-comment coupling that the mark-read-on-delivery approach
// suffered.
let mut delivered: HashMap<u64, String> = crate::harness_state::read_forge_cursor();
if !delivered.is_empty() {
info!(
entries = delivered.len(),
"forge_notify: restored delivery-dedupe cursor"
);
}
loop { loop {
interval.tick().await; interval.tick().await;
@ -1054,21 +1040,16 @@ async fn poll_once(
let notifications: Vec<PolledNotification> = let notifications: Vec<PolledNotification> =
values.into_iter().filter_map(parse_notification).collect(); values.into_iter().filter_map(parse_notification).collect();
// Tracks whether the dedupe cursor changed this poll (a new delivery
// recorded, or the prune below dropped now-read threads) so we only
// rewrite the on-disk cursor when there's something to persist.
let mut cursor_dirty = false;
for notif in &notifications { for notif in &notifications {
let Some(id) = notif.thread.id.and_then(|id| u64::try_from(id).ok()) else { let Some(id) = notif.thread.id.and_then(|id| u64::try_from(id).ok()) else {
continue; continue;
}; };
// Delivery-dedupe: we no longer mark threads read on delivery, so // In-process delivery-dedupe: guards against re-firing a wake for a
// an unread notification reappears in every `?all=false` poll. // thread whose mark-read (below) transiently failed and so still
// Skip it silently unless its `updated_at` advanced since the // shows up unread in the next `?all=false` poll. Skip unless its
// version we last delivered a wake for (i.e. genuinely new // `updated_at` advanced since the version we last delivered (i.e.
// activity). See the `delivered` cursor note in `run`. // genuinely new activity). See the `delivered` note in `run`.
let updated_at = notif.updated_at.clone(); let updated_at = notif.updated_at.clone();
if !should_deliver(delivered, id, &updated_at) { if !should_deliver(delivered, id, &updated_at) {
debug!(%id, "forge_notify: skipping (already delivered this version)"); debug!(%id, "forge_notify: skipping (already delivered this version)");
@ -1093,15 +1074,19 @@ async fn poll_once(
match deliver_result { match deliver_result {
Ok(()) => { Ok(()) => {
debug!(%id, "forge_notify: delivered"); debug!(%id, "forge_notify: delivered");
// Record the delivered version in the dedupe cursor INSTEAD // Mark the thread read on forge immediately after a
// of marking the thread read. Leaving it unread is // successful broker delivery. The broker inbox is the
// deliberate: the hive-forge read-before-comment guard keys // durable work queue now (each row has its own ack
// off forge's own unread-state, and the agent reading the // lifecycle), so the forge unread flag no longer needs to
// thread via the CLI is what marks it read. Recorded only // track agent processing — clearing it on delivery keeps
// here in the Ok arm — a failed delivery leaves the cursor // forge's unread set tiny by construction, so a container
// untouched, so it re-delivers next tick. // rebuild re-scan finds nothing stale to re-deliver. The
// in-memory `delivered` entry below is only a within-process
// guard so a transient mark-read failure doesn't re-fire the
// wake next tick; it is deliberately NOT persisted — forge's
// own read-state is the cross-rebuild source of truth.
mark_read(forge, id).await;
delivered.insert(id, updated_at); delivered.insert(id, updated_at);
cursor_dirty = true;
} }
Err(e) => { Err(e) => {
warn!(%id, error = ?e, "forge_notify: deliver failed — leaving unread"); warn!(%id, error = ?e, "forge_notify: deliver failed — leaving unread");
@ -1109,39 +1094,24 @@ async fn poll_once(
} }
} }
// Prune the dedupe cursor down to the threads still present in this // Prune the in-process dedupe map to the threads still present in this
// poll's unread set. Once the agent reads a thread (marking it read // poll's unread set. A delivered thread is marked read above, so it
// via the CLI) it drops out of `?all=false`, so its cursor entry is // drops out of `?all=false` next poll and its entry becomes dead
// dead weight; dropping it bounds the map to the current unread size. // weight; dropping it bounds the map to the current unread size.
// If such a thread later goes unread again it carries a fresh
// `updated_at` and re-delivers correctly.
// //
// This retain IS the cursor's size bound: `current_ids` comes from a // `current_ids` comes from a single `limit=UNREAD_FETCH_LIMIT` page, so
// single `limit=UNREAD_FETCH_LIMIT` page, so the persisted cursor can // the map can never exceed that many entries — the assert makes that
// never exceed that many entries — it tracks the unread *window*, not // invariant loud in tests/dev if a future pagination change breaks it.
// the all-time notification count. The assert makes the invariant
// loud in tests/dev if a future pagination change silently breaks it.
let current_ids: HashSet<u64> = notifications let current_ids: HashSet<u64> = notifications
.iter() .iter()
.filter_map(|n| n.thread.id.and_then(|id| u64::try_from(id).ok())) .filter_map(|n| n.thread.id.and_then(|id| u64::try_from(id).ok()))
.collect(); .collect();
let before_prune = delivered.len();
delivered.retain(|id, _| current_ids.contains(id)); delivered.retain(|id, _| current_ids.contains(id));
debug_assert!( debug_assert!(
delivered.len() <= UNREAD_FETCH_LIMIT, delivered.len() <= UNREAD_FETCH_LIMIT,
"dedupe cursor exceeded the fetch window ({} > {UNREAD_FETCH_LIMIT})", "in-process dedupe map exceeded the fetch window ({} > {UNREAD_FETCH_LIMIT})",
delivered.len(), delivered.len(),
); );
if delivered.len() != before_prune {
cursor_dirty = true;
}
// Flush the cursor to the consolidated state file only when it
// changed, so a rebuild/restart reloads it instead of re-delivering
// the whole unread backlog.
if cursor_dirty {
crate::harness_state::write_forge_cursor(delivered);
}
} }
/// Whether a notification should be delivered as a wake given the /// Whether a notification should be delivered as a wake given the
@ -1153,11 +1123,13 @@ fn should_deliver(delivered: &HashMap<u64, String>, id: u64, updated_at: &str) -
} }
/// Mark a notification thread as read. Best-effort — logs on failure but /// Mark a notification thread as read. Best-effort — logs on failure but
/// does not abort the poll loop. Called only on the self-echo path (the /// does not abort the poll loop. Called on two paths: right after a
/// agent's own comment/review/creation writes) — delivered threads are /// successful broker delivery (so forge's unread set stays tiny and a
/// deliberately left unread for the read-before-comment guard, and a failed /// rebuild can't re-deliver), and on the self-echo path (the agent's own
/// delivery is left unread + out of the dedupe cursor so it resurfaces on the /// comment/review/creation writes, delivered nowhere). A failed delivery
/// next poll tick. /// leaves the thread unread + out of the in-process dedupe map so it
/// resurfaces on the next poll tick; a transient mark-read failure after a
/// good delivery is caught by that same dedupe map (no duplicate wake).
async fn mark_read(forge: &Forgejo, id: u64) { async fn mark_read(forge: &Forgejo, id: u64) {
let Ok(thread_id) = i64::try_from(id) else { let Ok(thread_id) = i64::try_from(id) else {
// Thread ids originate from `i64` in the poll parse, so an // Thread ids originate from `i64` in the poll parse, so an

66
hive-agent/src/harness_state.rs
View file

@ -140,7 +140,7 @@ pub(crate) fn read_harness_state() -> (bool, bool, Option<String>) {
} }
/// Write the turn-loop's harness state fields via a read-modify-write so /// Write the turn-loop's harness state fields via a read-modify-write so
/// any other writer's fields (e.g. `forge_notify`'s `forge_cursor`) survive. /// any other writer's fields in the consolidated state file survive.
/// Pass `active_model: Some(s)` to update the resolved model (surfaced in /// Pass `active_model: Some(s)` to update the resolved model (surfaced in
/// the dashboard badge); `None` leaves the stored value untouched. /// the dashboard badge); `None` leaves the stored value untouched.
pub(crate) fn write_harness_state( pub(crate) fn write_harness_state(
@ -160,40 +160,6 @@ pub(crate) fn write_harness_state(
write_harness_json(&v); write_harness_json(&v);
} }
/// Parse the `forge_notify` delivery-dedupe cursor (notification thread id
/// -> last-delivered `updated_at`) out of a harness-state JSON value.
/// Empty when the field is absent (first boot) or malformed.
fn forge_cursor_from_json(v: &serde_json::Value) -> std::collections::HashMap<u64, String> {
v.get("forge_cursor")
.cloned()
.and_then(|c| serde_json::from_value(c).ok())
.unwrap_or_default()
}
/// Restore the `forge_notify` delivery-dedupe cursor from the consolidated
/// state file so a container rebuild/restart doesn't re-deliver the whole
/// currently-unread backlog.
pub fn read_forge_cursor() -> std::collections::HashMap<u64, String> {
forge_cursor_from_json(&read_harness_json())
}
/// Persist the `forge_notify` delivery-dedupe cursor into the consolidated
/// state file, read-modify-write under the shared lock so the turn-loop's
/// own fields survive. Best-effort: a serialize failure is a no-op.
pub fn write_forge_cursor<S: std::hash::BuildHasher>(
cursor: &std::collections::HashMap<u64, String, S>,
) {
let Ok(value) = serde_json::to_value(cursor) else {
return;
};
let _guard = HARNESS_JSON_LOCK
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let mut v = read_harness_json();
v["forge_cursor"] = value;
write_harness_json(&v);
}
/// Compiled-in fallback model used when neither `HIVE_DEFAULT_MODEL` nor a /// Compiled-in fallback model used when neither `HIVE_DEFAULT_MODEL` nor a
/// persisted runtime override is present. /// persisted runtime override is present.
pub const DEFAULT_MODEL: &str = "haiku"; pub const DEFAULT_MODEL: &str = "haiku";
@ -280,35 +246,7 @@ pub fn context_window_tokens(model: &str) -> u64 {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::{DEFAULT_EFFORT, EFFORT_LEVELS, forge_cursor_from_json, is_valid_effort}; use super::{DEFAULT_EFFORT, EFFORT_LEVELS, is_valid_effort};
use serde_json::json;
#[test]
fn forge_cursor_absent_field_is_empty() {
// First boot / a state file that only carries the turn-loop fields:
// no cursor yet, so we re-deliver the currently-unread set once.
assert!(forge_cursor_from_json(&json!({ "rate_limited": false })).is_empty());
}
#[test]
fn forge_cursor_malformed_field_is_empty() {
// A wrong-typed / corrupt cursor degrades to empty rather than
// aborting the poller.
assert!(forge_cursor_from_json(&json!({ "forge_cursor": "nonsense" })).is_empty());
}
#[test]
fn forge_cursor_roundtrips_u64_keys() {
// serde_json stringifies integer map keys; confirm the u64 thread
// ids the cursor is keyed on survive the JSON round-trip.
let v = json!({
"forge_cursor": { "42": "2026-06-22T16:00:00Z", "99": "2026-06-22T17:30:00Z" }
});
let cursor = forge_cursor_from_json(&v);
assert_eq!(cursor.len(), 2);
assert_eq!(cursor.get(&42), Some(&"2026-06-22T16:00:00Z".to_owned()));
assert_eq!(cursor.get(&99), Some(&"2026-06-22T17:30:00Z".to_owned()));
}
#[test] #[test]
fn effort_validation_accepts_only_known_levels() { fn effort_validation_accepts_only_known_levels() {