vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

swarm-controller: bound forge calls in job executors

Browse source 
forgejo-api's client has no request timeout, so a forge that accepts a
connection and never answers held a forge-calling job node forever:
the node never reached Done or Failed, and kept its executor task.

Every executor arm that awaits the forge now runs its work under the
same deadline the periodic passes use. On elapsing, the node fails
through its existing Outcome::Failed path with
"<node>: no answer from the forge within 120s".

Nodes: create_repo, create_forge_user, add_repo_member,
grant_docs_read, init_agent_config_repo, mint_agent_forge_token,
mint_hive_forge_token.

The bound is per node, not per request: the nodes make 1 to about 15
sequential forge and store requests each (mint_hive_forge_token the
most), and the requests live inside forge::Client methods the periodic
passes share. One 2-minute bound covers both, so #5065's
within_pass_deadline / PASS_DEADLINE become within_forge_deadline /
FORGE_DEADLINE, with a log line that no longer says "retrying next
tick" (true of a pass, not of a node).

The InitAgentConfigRepo arm is lifted out of run_swarm_node, like its
siblings, to keep that function under clippy::too_many_lines.

Refs #5066
This commit is contained in:
atlas 2026-10-09 20:00:19 +02:00 • committed by mara
commit 6f8a6ab8e0
8 changed files with 190 additions and 50 deletions
Download patch file Download diff file Expand all files Collapse all files

2
swarm-controller/src/forge/agent_token.rs
View file

@ -370,7 +370,7 @@ pub fn spawn(client: Arc<Client>, enqueue: impl Fn(Vec<String>) + Send + 'static
let mut ticker = tokio::time::interval(RECONCILE_INTERVAL);
loop {
ticker.tick().await;
let Some(outcome) = super::within_pass_deadline("agent forge tokens", async {
let Some(outcome) = super::within_forge_deadline("agent forge tokens", async {
let observed = client.observe_all().await?;
let legacy = super::legacy_tokens::sweepable(&observed);
client.sweep_legacy_tokens(&legacy).await;

2
swarm-controller/src/forge/docs_grant.rs
View file

@ -185,7 +185,7 @@ pub fn spawn(client: Arc<Client>, enqueue: impl Fn(Vec<String>) + Send + 'static
let mut ticker = tokio::time::interval(RECONCILE_INTERVAL);
loop {
ticker.tick().await;
let Some(agents) = super::within_pass_deadline("docs grants", async {
let Some(agents) = super::within_forge_deadline("docs grants", async {
run_pass(&client, read_roster().await).await
})
.await

2
swarm-controller/src/forge/hive_token.rs
View file

@ -387,7 +387,7 @@ pub fn spawn(
loop {
ticker.tick().await;
let Some(outcome) =
super::within_pass_deadline("hive forge tokens", client.observe_hives(&hives))
super::within_forge_deadline("hive forge tokens", client.observe_hives(&hives))
.await
else {
continue;

2
swarm-controller/src/forge/objects.rs
View file

@ -1083,7 +1083,7 @@ pub fn spawn(client: Arc<Client>) {
loop {
ticker.tick().await;
let Some(outcome) =
super::within_pass_deadline("swarm forge objects", client.reconcile(&desired))
super::within_forge_deadline("swarm forge objects", client.reconcile(&desired))
.await
else {
continue;

2
swarm-controller/src/forge/runner_token.rs
View file

@ -77,7 +77,7 @@ pub fn spawn(client: Arc<Client>) {
let mut last_written: Option<String> = None;
loop {
ticker.tick().await;
let Some(outcome) = super::within_pass_deadline(
let Some(outcome) = super::within_forge_deadline(
"runner registration token",
pass(&client, last_written.as_deref()),
)