Cap agentic tool-loop iterations per turn, so a turn is not bounded only by the 5-minute wall clock

coilyco-ops commented

2026-08-19 14:53:10 +00:00

Member

What

A turn has no bound on how many model completions it may run. The only limit is SIRENS_ECHO_REQUEST_TIMEOUT, set to 5m on every lane. A turn that keeps deciding to call another tool runs until that wall clock, so worst-case turn latency is pinned at roughly 300s rather than at anything the harness chose.

Evidence

Measured on the sirens-dowel lane from SigNoz traces on 2026-08-19, 24h window.

Turn duration tracks completion count almost exactly. model.chat p50 is 5.1s to 6.0s, stable every hour for 12h, and community.turn duration is close to that multiplied by the number of loop iterations.
Baseline hours run 2 to 4 completions per turn.
One recent hour ran 79 completions across 5 turns, about 16 per turn, and turn p95 in that hour hit 291s.
The heaviest single turn in the window ran 36 completions and took 235.9s.
Across 24h, 11 of 117 turns exceeded 90s, and community.turn p95 was 181s against a p50 of 17.2s.

The model backend is not the problem. Per-completion latency held steady across the whole window including the worst hour, so the tail is iteration count, not model slowness.

Lane comparison over the same window, both lanes on sirens-echo/deepseek through Agent Proxy:

sirens-dowel - 3.65 completions per turn, p50 17.2s, p95 181s
sirens-deep - 2.07 completions per turn, p50 33.8s, p95 53s

Dowel's median is better than Deep's. Only its tail is bad, which is the signature of an unbounded loop rather than a slow component.

Why a timeout is not the fix

Lowering SIRENS_ECHO_REQUEST_TIMEOUT bounds the tail by killing turns. At 90s it would have dropped 11 of 117 turns over 24h, roughly 1 in 11, and the turns it drops are the hard investigative ones. That trades a slow answer for no answer.

Proposal

Add a per-turn iteration cap, exposed as an environment knob so a lane can tune it alongside its roster. On reaching the cap the harness stops calling tools and asks the model to answer from what it already gathered, which yields a degraded answer rather than a dropped turn.

I checked every SIRENS_ECHO_* name referenced in the deploy repo. There is no iteration, step, or tool-budget knob today, only SIRENS_ECHO_REQUEST_TIMEOUT and SIRENS_ECHO_QUEUE_TIMEOUT.

Note on lane shape

Not confirmed in code, recorded as inference. Dowel loops more than Deep plausibly because its roster is investigation tooling (playwright, signoz, temporal, exa) where each result invites another call, and because it runs SIRENS_ECHO_ROLE=engineer, whose bundle is curious, meticulous, and tenacious. A cap makes the lane's behavior bounded regardless of which of those is driving it.

Reproducing

Query SigNoz traces with service.name = 'sirens-dowel', then group name = 'model.chat' by trace_id ordered by count descending to see the per-turn distribution, and compare against name = 'community.turn' durations over the same window.

## What A turn has no bound on how many model completions it may run. The only limit is `SIRENS_ECHO_REQUEST_TIMEOUT`, set to `5m` on every lane. A turn that keeps deciding to call another tool runs until that wall clock, so worst-case turn latency is pinned at roughly 300s rather than at anything the harness chose. ## Evidence Measured on the `sirens-dowel` lane from SigNoz traces on 2026-08-19, 24h window. * Turn duration tracks completion count almost exactly. `model.chat` p50 is 5.1s to 6.0s, stable every hour for 12h, and `community.turn` duration is close to that multiplied by the number of loop iterations. * Baseline hours run 2 to 4 completions per turn. * One recent hour ran 79 completions across 5 turns, about 16 per turn, and turn p95 in that hour hit 291s. * The heaviest single turn in the window ran 36 completions and took 235.9s. * Across 24h, 11 of 117 turns exceeded 90s, and `community.turn` p95 was 181s against a p50 of 17.2s. The model backend is not the problem. Per-completion latency held steady across the whole window including the worst hour, so the tail is iteration count, not model slowness. Lane comparison over the same window, both lanes on `sirens-echo/deepseek` through Agent Proxy: * `sirens-dowel` - 3.65 completions per turn, p50 17.2s, p95 181s * `sirens-deep` - 2.07 completions per turn, p50 33.8s, p95 53s Dowel's median is better than Deep's. Only its tail is bad, which is the signature of an unbounded loop rather than a slow component. ## Why a timeout is not the fix Lowering `SIRENS_ECHO_REQUEST_TIMEOUT` bounds the tail by killing turns. At 90s it would have dropped 11 of 117 turns over 24h, roughly 1 in 11, and the turns it drops are the hard investigative ones. That trades a slow answer for no answer. ## Proposal Add a per-turn iteration cap, exposed as an environment knob so a lane can tune it alongside its roster. On reaching the cap the harness stops calling tools and asks the model to answer from what it already gathered, which yields a degraded answer rather than a dropped turn. I checked every `SIRENS_ECHO_*` name referenced in the deploy repo. There is no iteration, step, or tool-budget knob today, only `SIRENS_ECHO_REQUEST_TIMEOUT` and `SIRENS_ECHO_QUEUE_TIMEOUT`. ## Note on lane shape Not confirmed in code, recorded as inference. Dowel loops more than Deep plausibly because its roster is investigation tooling (playwright, signoz, temporal, exa) where each result invites another call, and because it runs `SIRENS_ECHO_ROLE=engineer`, whose bundle is curious, meticulous, and tenacious. A cap makes the lane's behavior bounded regardless of which of those is driving it. ## Reproducing Query SigNoz traces with `service.name = 'sirens-dowel'`, then group `name = 'model.chat'` by `trace_id` ordered by count descending to see the per-turn distribution, and compare against `name = 'community.turn'` durations over the same window.

coilyco-ops commented

2026-08-19 14:59:28 +00:00

Author

Member

Correcting the "Note on lane shape" section above. It guessed that Dowel's investigation-heavy roster invites long chains, and marked that as inference. Measuring it shows the guess was wrong, which strengthens rather than weakens the case for a cap.

Tool-call concentration across 117 turns in the same 24h window:

signoz - 39 calls across 10 turns (8.5% of turns), with 69% of them in just 3 turns
playwright - 52 calls across 14 turns (12% of turns), with 58% of them in a single turn

Neither server is reached for speculatively. Both are used only when a turn is explicitly pointed at them, and playwright is the more concentrated of the two.

The decisive check is the heaviest turn in the window, at 36 completions and 235.9s. It contains zero signoz calls. Its 35 tool calls are 30 playwright plus 4 web search plus 1 scratch. Separately, the only turn that hit the 5-minute timeout and errored is the signoz-heaviest turn in the window, at 13 signoz calls.

Ranking the top ten turns by tool-call volume shows a different tool mix in each. There is no consistent tool signature to the heavy tail.

What this means for the fix: the long chains are a property of question difficulty, not of roster shape. Trimming a lane's roster would remove capability without moving the tail, so an iteration cap is the correct lever rather than roster curation. It also means the cap belongs in the harness for every lane rather than being tuned per lane to compensate for which servers a lane happens to carry, though a per-lane override is still worth having.

Correcting the "Note on lane shape" section above. It guessed that Dowel's investigation-heavy roster invites long chains, and marked that as inference. Measuring it shows the guess was wrong, which strengthens rather than weakens the case for a cap. Tool-call concentration across 117 turns in the same 24h window: * signoz - 39 calls across 10 turns (8.5% of turns), with 69% of them in just 3 turns * playwright - 52 calls across 14 turns (12% of turns), with 58% of them in a single turn Neither server is reached for speculatively. Both are used only when a turn is explicitly pointed at them, and playwright is the more concentrated of the two. The decisive check is the heaviest turn in the window, at 36 completions and 235.9s. It contains zero signoz calls. Its 35 tool calls are 30 playwright plus 4 web search plus 1 scratch. Separately, the only turn that hit the 5-minute timeout and errored is the signoz-heaviest turn in the window, at 13 signoz calls. Ranking the top ten turns by tool-call volume shows a different tool mix in each. There is no consistent tool signature to the heavy tail. What this means for the fix: the long chains are a property of question difficulty, not of roster shape. Trimming a lane's roster would remove capability without moving the tail, so an iteration cap is the correct lever rather than roster curation. It also means the cap belongs in the harness for every lane rather than being tuned per lane to compensate for which servers a lane happens to carry, though a per-lane override is still worth having.

coilyco-ops referenced this issue from a commit

2026-08-19 15:30:30 +00:00

feat(proxy): bound a turn, not just each completion in it

coilyco-ops referenced this issue from a pull request that will close it,

2026-08-19 15:31:00 +00:00

feat(proxy): bound a turn, not just each completion in it #1077