feat(proxy): bound a turn, not just each completion in it #1077

Merged

coilysiren merged 1 commit from aos/claude/turn-iteration-cap into main

2026-08-19 15:32:20 +00:00

coilyco-ops commented

2026-08-19 15:31:00 +00:00

Member

Closes #1076.

What the measurement changed about the issue

The issue proposed a per-turn iteration cap with an env knob, degrading to an
answer from gathered results rather than dropping the turn. That mechanism
already existed as SIRENS_ECHO_TOOL_ROUNDS / model_budget.tool_rounds plus
the toolsSpent path, and the traces show it is honoured: model.round never
passed 15 across the whole 24h window, exactly the last round of dowel's
16-call ceiling. The issue's grep found no knob because it searched the deploy
repo rather than the harness.

The tail has a different cause. The heaviest turn ran 36 model.chat spans
in 235.9s, and its rounds read 0-8 three times each and taper to 13 once. That
is three completions of 14, 13, and 9 rounds, not one loop that ran long,
and mcp.tools.list appears three times in the same trace, one per completion
that opened a roster. A turn runs the content gate, the answer, and the filing
check as separate Complete calls, and each one opened the ceiling again. The
per-completion bound multiplied instead of binding. Nothing counted the turn.

Retry amplification was ruled out: every failure path marks its span errored,
and only one errored model.chat span exists in the window.

The change

SIRENS_ECHO_TURN_MODEL_CALLS (24, per lane model_budget.turn_model_calls)
rides the turn context and is spent by every completion under it.

It gates investigation and never the answer. Under two calls a tool round no
longer fits, so the tools are withdrawn and the existing spent-budget notice is
appended, logged as model.turn.budget.spent, while every completion still buys
its final call. A starved content gate or filing check fails a turn outright,
which is a worse outcome than a turn answering from partial evidence and saying
which parts it could not determine. An allowance under two would make the lane
silently toolless, so configuration refuses it.

Only an ingress turn installs an allowance. The evaluation board, the bridge
runner, and the rate lane call Complete outside a turn and carry none, and a
nil budget affords everything, so they are unchanged.

Against the measured window, 24 would have bounded the 36-call and 26-call turns
and left the 20/17/15-call turns untouched. The p50 of 3.65 completions per turn
is nowhere near it.

Notes for review

docs/FEATURES.md is not updated. It sits at exactly its 120-line /
7924-char caps on main, and docs/ is at its 40-page cap, so neither a new
page nor a new bullet fits without evicting existing content. On a fair
reading this rescopes an existing bound rather than shipping a new capability,
which is the case FEATURES says to leave alone. The prose landed in
docs/sirens-echo-turn-stages.md, already linked from the FEATURES index.
Say the word if you would rather make room for an entry.
agent/rendered/knobs.txt is regenerated by its own test.

go build, go vet, go test ./..., and pre-commit run --all-files are all
green.

Closes #1076. ## What the measurement changed about the issue The issue proposed a per-turn iteration cap with an env knob, degrading to an answer from gathered results rather than dropping the turn. That mechanism already existed as `SIRENS_ECHO_TOOL_ROUNDS` / `model_budget.tool_rounds` plus the `toolsSpent` path, and the traces show it is honoured: `model.round` never passed **15** across the whole 24h window, exactly the last round of dowel's 16-call ceiling. The issue's grep found no knob because it searched the deploy repo rather than the harness. The tail has a different cause. The heaviest turn ran **36** `model.chat` spans in 235.9s, and its rounds read 0-8 three times each and taper to 13 once. That is **three completions** of 14, 13, and 9 rounds, not one loop that ran long, and `mcp.tools.list` appears three times in the same trace, one per completion that opened a roster. A turn runs the content gate, the answer, and the filing check as separate `Complete` calls, and each one opened the ceiling again. The per-completion bound multiplied instead of binding. Nothing counted the turn. Retry amplification was ruled out: every failure path marks its span errored, and only one errored `model.chat` span exists in the window. ## The change `SIRENS_ECHO_TURN_MODEL_CALLS` (`24`, per lane `model_budget.turn_model_calls`) rides the turn context and is spent by every completion under it. **It gates investigation and never the answer.** Under two calls a tool round no longer fits, so the tools are withdrawn and the existing spent-budget notice is appended, logged as `model.turn.budget.spent`, while every completion still buys its final call. A starved content gate or filing check fails a turn outright, which is a worse outcome than a turn answering from partial evidence and saying which parts it could not determine. An allowance under two would make the lane silently toolless, so configuration refuses it. Only an ingress turn installs an allowance. The evaluation board, the bridge runner, and the rate lane call `Complete` outside a turn and carry none, and a nil budget affords everything, so they are unchanged. Against the measured window, 24 would have bounded the 36-call and 26-call turns and left the 20/17/15-call turns untouched. The p50 of 3.65 completions per turn is nowhere near it. ## Notes for review - `docs/FEATURES.md` is **not** updated. It sits at exactly its 120-line / 7924-char caps on main, and `docs/` is at its 40-page cap, so neither a new page nor a new bullet fits without evicting existing content. On a fair reading this rescopes an existing bound rather than shipping a new capability, which is the case FEATURES says to leave alone. The prose landed in `docs/sirens-echo-turn-stages.md`, already linked from the FEATURES index. Say the word if you would rather make room for an entry. - `agent/rendered/knobs.txt` is regenerated by its own test. `go build`, `go vet`, `go test ./...`, and `pre-commit run --all-files` are all green.

coilyco-ops added 1 commit

2026-08-19 15:31:00 +00:00

feat(proxy): bound a turn, not just each completion in it

ci / image-build (pull_request) Successful in 38s

Details

ci / test (pull_request) Successful in 1m3s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

35c721340a

ModelBudget bounds one Complete call and a turn makes several: the content
gate, the answer, and the filing check each open the ceiling again, so it
multiplied rather than bound.

Measured on sirens-dowel over 24h: model.round never passed 15, the last
round of a 16-call ceiling, while the heaviest turn ran 36 model.chat spans
in 235.9s. Its rounds read 0-8 three times and taper to 13 once, so that
turn is three completions of 14, 13, and 9 rather than one long loop.

SIRENS_ECHO_TURN_MODEL_CALLS (24, per lane model_budget.turn_model_calls)
rides the turn context and is spent by every completion under it. It gates
investigation and never the answer: under two calls a tool round no longer
fits, so tools are withdrawn and the existing spent-budget notice asks for
an answer from what was gathered. A starved content gate or filing check
fails the turn outright, which is worse than a degraded answer.

Closes #1076

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

coilyco-ops requested review from coilysiren

2026-08-19 15:31:00 +00:00