fix: bound tool results and escalate the completion budget #100

Merged

coilysiren merged 1 commit from fix/tool-inflated-completion-budget into main

2026-08-11 06:10:53 +00:00

coilyco-ops commented

2026-08-11 06:03:20 +00:00

Member

Fixes the 20% turn failure rate reported at #86.

The chain

Tool results re-inject into the next request, which took a 6k prompt past 47k on four parallel Eco calls. The model then spent the whole fixed 900-token completion budget on reasoning_content and returned empty content with finish_reason: length. The single repair retried at the same budget and hit the same wall.

The trigger was never question difficulty. It was whether the turn reached for a tool, which is why the failure rate was worst on ordinary member questions and zero on trivial and identity checks.

Why nothing reported it

A truncated completion is a valid response at the transport layer, so Agent Proxy and LiteLLM both logged outcome=ok.

Worse, the runtime could not see it either: finish_reason was not captured from the response at all, so a truncated completion was indistinguishable from a finished one that happened to be empty. Capturing it is the smallest part of this change and the one that makes the rest possible.

Two bounds, because cause and symptom differ

Tool results are capped before re-injection. Only the copy entering the prompt is bounded, and the model sees a truncation marker. The full result is kept for grounding validation, so bounding cannot make the runtime accept an action claim it should reject.

The completion budget escalates on a completion that is both truncated and empty, from 900 to a cap of 3600 across two raises, then fails with an error naming the truncation rather than a generic contract failure.

Raising alone would treat the symptom, since a larger prompt finds the new ceiling too. Bounding alone leaves any turn that still overruns failing at a fixed wall.

Truncated output that is not empty is a usable answer and does not raise. That case is tested.

Negative-tested

Disabling the escalation fails both budget tests. Disabling the bound fails the tool-result test. Verified in both directions rather than assumed.

Timing

Worth landing before the composed role bundle from #98, since a role identity makes the system prompt materially larger and pushes directly into this failure.

Not addressed here

The issue also notes latency degrading under 4-way concurrency, p95 25.85s and max 43.44s. That is a separate concern from the failure rate and wants its own measurement against current main.

Verification

pre-commit run --all-files exit 0, go vet, and go test -count=1 -race ./... all pass.

Fixes the 20% turn failure rate reported at https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/86. ## The chain Tool results re-inject into the next request, which took a 6k prompt past 47k on four parallel Eco calls. The model then spent the whole fixed 900-token completion budget on `reasoning_content` and returned empty content with `finish_reason: length`. The single repair retried at the same budget and hit the same wall. The trigger was never question difficulty. It was whether the turn reached for a tool, which is why the failure rate was worst on ordinary member questions and zero on trivial and identity checks. ## Why nothing reported it A truncated completion is a valid response at the transport layer, so Agent Proxy and LiteLLM both logged `outcome=ok`. Worse, the runtime could not see it either: **`finish_reason` was not captured from the response at all**, so a truncated completion was indistinguishable from a finished one that happened to be empty. Capturing it is the smallest part of this change and the one that makes the rest possible. ## Two bounds, because cause and symptom differ **Tool results are capped before re-injection.** Only the copy entering the prompt is bounded, and the model sees a truncation marker. The full result is kept for grounding validation, so bounding cannot make the runtime accept an action claim it should reject. **The completion budget escalates** on a completion that is both truncated and empty, from 900 to a cap of 3600 across two raises, then fails with an error naming the truncation rather than a generic contract failure. Raising alone would treat the symptom, since a larger prompt finds the new ceiling too. Bounding alone leaves any turn that still overruns failing at a fixed wall. Truncated output that is **not** empty is a usable answer and does not raise. That case is tested. ## Negative-tested Disabling the escalation fails both budget tests. Disabling the bound fails the tool-result test. Verified in both directions rather than assumed. ## Timing Worth landing before the composed role bundle from https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/98, since a role identity makes the system prompt materially larger and pushes directly into this failure. ## Not addressed here The issue also notes latency degrading under 4-way concurrency, p95 25.85s and max 43.44s. That is a separate concern from the failure rate and wants its own measurement against current main. ## Verification `pre-commit run --all-files` exit 0, `go vet`, and `go test -count=1 -race ./...` all pass.

coilyco-ops added 1 commit

2026-08-11 06:03:21 +00:00

fix: bound tool results and escalate the completion budget

ci / test (pull_request) Successful in 35s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

b0beff8a66

Twenty percent of turns failed, worst on ordinary member questions, because
the trigger was not difficulty but whether the turn reached for a tool.

Tool results re-inject into the next request, which took a 6k prompt past
47k on four parallel Eco calls. The model then spent the whole fixed 900
token completion budget on reasoning_content and returned empty content
with finish_reason length. The single repair retried at the same budget
and hit the same wall.

Nothing upstream reported it, because a truncated completion is a valid
response at the transport layer. Agent Proxy and LiteLLM both logged
outcome=ok.

The runtime could not even see the problem: finish_reason was not captured
from the response, so a truncated completion was indistinguishable from a
finished one that happened to be empty.

Two bounds, because the cause and the symptom are different.

Tool results are capped before re-injection. Only the copy entering the
prompt is bounded and the model sees a truncation marker. The full result
is kept for grounding validation, so bounding cannot make the runtime
accept an action claim it should reject.

The completion budget escalates on a completion that is truncated and
empty, from 900 to a cap of 3600 across two raises, then fails with an
error naming the truncation. Raising alone would treat the symptom, since
a larger prompt finds the new ceiling too. Truncated output that is not
empty is a usable answer and does not raise.

Each half is negative-tested. Disabling the escalation fails the two
budget tests, and disabling the bound fails the tool-result test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

coilyco-ops requested review from coilysiren

2026-08-11 06:03:21 +00:00