fix(proxy): say how much reasoning a spent budget bought #335

Merged

coilysiren merged 2 commits from fix/budget-failure-names-the-reasoning-spend into main

2026-08-13 10:22:21 +00:00

coilyco-ops commented

2026-08-13 10:08:18 +00:00

Member

closes #334

The engineering slice of #325. The tuning and the instrument halves stay open there.

The gap

A knowledge question returned empty content 4 of 10 times after burning 3600 tokens and two raises. Every occurrence rendered identically, so two conditions wanting opposite responses were indistinguishable:

condition	reasoning content	response
the model thought and ran out	large	the ceiling is too low
the model produced nothing	empty	not a budget problem at all

chatResponseMessage already parsed reasoning_content and nothing read it. The evidence separating the two was discarded at the moment of failure.

What landed

The truncation failure and model.budget.raised both carry the reasoning size. A byte count, never the text — reasoning content is model output and the turn logger carries no model bodies, so a test asserts the message leaks none of it.

formatBudgetExhausted is extracted so the message is testable rather than only reachable through a live model.

What is deliberately unchanged

The escalation is not broken and I checked before touching it. 900 to 1800 to 3600 across two raises is what its documentation says, and truncated() correctly requires a length finish and empty content and no tool calls, so a length finish that still produced a usable answer is not discarded. Tests pin all three.

maxCompletionTokens, budgetRaisesAllowed, completionBudgetStep. Whether 3600 is enough for a reasoning model is a cost and latency decision against a live route. Raising a ceiling because a measurement was uncomfortable is how a budget stops meaning anything.

One correction on my own commit

The commit message says closes #325, which is too broad — 325 has three halves and this is one. I filed #334 as the slice and this body closes that instead, which is the partial-delivery rule from #330 that I documented an hour ago and then broke on the very next commit.

ward exec gate green: build, policy-check, vet, test, test-skips, pre-commit.

closes #334 The engineering slice of https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/325. **The tuning and the instrument halves stay open there.** ## The gap A knowledge question returned empty content 4 of 10 times after burning 3600 tokens and two raises. Every occurrence rendered identically, so two conditions wanting opposite responses were indistinguishable: | condition | reasoning content | response | | --- | --- | --- | | the model thought and ran out | large | the ceiling is too low | | the model produced nothing | empty | not a budget problem at all | `chatResponseMessage` already parsed `reasoning_content` and nothing read it. The evidence separating the two was discarded at the moment of failure. ## What landed The truncation failure and `model.budget.raised` both carry the reasoning size. A **byte count, never the text** — reasoning content is model output and the turn logger carries no model bodies, so a test asserts the message leaks none of it. `formatBudgetExhausted` is extracted so the message is testable rather than only reachable through a live model. ## What is deliberately unchanged **The escalation is not broken and I checked before touching it.** 900 to 1800 to 3600 across two raises is what its documentation says, and `truncated()` correctly requires a length finish **and** empty content **and** no tool calls, so a length finish that still produced a usable answer is not discarded. Tests pin all three. **`maxCompletionTokens`, `budgetRaisesAllowed`, `completionBudgetStep`.** Whether 3600 is enough for a reasoning model is a cost and latency decision against a live route. Raising a ceiling because a measurement was uncomfortable is how a budget stops meaning anything. ## One correction on my own commit The commit message says `closes #325`, which is too broad — 325 has three halves and this is one. I filed https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/334 as the slice and this body closes that instead, which is the partial-delivery rule from https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/330 that I documented an hour ago and then broke on the very next commit. `ward exec gate` green: build, policy-check, vet, test, test-skips, pre-commit.

coilyco-ops added 1 commit

2026-08-13 10:08:18 +00:00

fix(proxy): say how much reasoning a spent budget bought

ci / test (pull_request) Successful in 32s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

ci / image-build (pull_request) Successful in 20s

Details

7a4ea80f02

A knowledge question returned empty content 4 of 10 times after burning 3600
tokens and two raises. Every failure rendered identically, so two different
conditions were indistinguishable:

  the model thought and ran out    reasoning content large, budget too small
  the model produced nothing       reasoning content empty, not a budget problem

Those want opposite responses. One is a ceiling to raise, the other is a
question about the model, and the message could not tell them apart.

chatResponseMessage already parsed reasoning_content and nothing read it. The
failure and model.budget.raised now carry its size, so the next run says which
failure it was with a number rather than leaving it to be re-measured.

A byte count, never the text. Reasoning content is model output and the turn
logger carries no model bodies, so a test asserts the message leaks none of it.

The escalation itself is not changed and is not broken. 900 to 1800 to 3600
across two raises is what its documentation says, and truncated() correctly
requires a length finish with empty content and no tool calls, so a length
finish that still produced an answer stays usable. Tests pin all three.

Deliberately not touching maxCompletionTokens, budgetRaisesAllowed, or
completionBudgetStep. Whether 3600 is enough for a reasoning model answering a
knowledge question is a cost and latency decision against a live route, and
raising a ceiling because a measurement was uncomfortable is how a budget stops
meaning anything.

closes #325

Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

coilyco-ops requested review from coilysiren

2026-08-13 10:08:18 +00:00

coilyco-ops referenced this pull request

2026-08-13 10:10:42 +00:00

Echo's model route answers nothing in 120 seconds, so no Echo rate has ever been measured #324

coilyco-ops referenced this pull request

2026-08-13 10:12:21 +00:00

A knowledge question returns empty content 4 of 10 times after burning 3600 tokens, and every rate pack reports that as a clean pass #325

coilyco-ops added 1 commit

2026-08-13 10:21:14 +00:00

Merge branch 'main' into fix/budget-failure-names-the-reasoning-spend

ci / test (pull_request) Successful in 31s