feat(notice): cite the turn's trace when it ends in anything but success #340

Merged

coilysiren merged 1 commit from feat/notice-carries-the-trace-id into main

2026-08-13 10:22:46 +00:00

coilyco-ops commented

2026-08-13 10:18:58 +00:00

Member

closes #336

A member reporting "it said no and I do not know why" was handing over a report. Now they hand over a query.

> `turn timed out, retry shortly`
> `trace id 3dd883c6becba130e9f8b75e4593a94d`

Covers stage failures, timeouts, rate-limit cooldowns, queue sheds, and an undelivered reply. A successful reply never gains one, because the line is what marks a turn that did not succeed.

Two deviations from the request, both deliberate

trace id rather than trace ID:. Both lines take the notice alphabet, which admits [a-z0-9 ,./-] and nothing else, and a colon cannot survive it. I could have widened the alphabet for the label and did not: the strictness is what makes a notice recognisable at a glance as harness output rather than model output, and that is worth more than the punctuation. A test asserts both lines still match noticeShape.

Outside a span the line is omitted, not rendered blank. Not every refusal happens inside a turn span. A rate-limit shed can fire before one exists, and an empty identifier reads as a defect rather than as an absence.

What it cost

Nothing new is plumbed and no identifier is invented. The value is the turn's own trace, already carried by every metadata log inside it, so an operator pastes it into SigNoz and has the whole turn.

One thing worth a look, not blocking

A trace ID is not member data and is inert on its own, but it is an internal identifier appearing in a public channel. I treated that as intended because it is the point of the feature. If it should appear only in #bots and not on the HTTP profile, that is a one-line condition.

ward exec gate green: build, policy-check, vet, test, test-skips, pre-commit.

closes #336 A member reporting "it said no and I do not know why" was handing over a report. Now they hand over a query. ``` > `turn timed out, retry shortly` > `trace id 3dd883c6becba130e9f8b75e4593a94d` ``` Covers stage failures, timeouts, rate-limit cooldowns, queue sheds, and an undelivered reply. **A successful reply never gains one**, because the line is what marks a turn that did not succeed. ## Two deviations from the request, both deliberate **`trace id` rather than `trace ID:`.** Both lines take the notice alphabet, which admits `[a-z0-9 ,./-]` and nothing else, and a colon cannot survive it. I could have widened the alphabet for the label and did not: the strictness is what makes a notice recognisable at a glance as harness output rather than model output, and that is worth more than the punctuation. A test asserts **both** lines still match `noticeShape`. **Outside a span the line is omitted, not rendered blank.** Not every refusal happens inside a turn span. A rate-limit shed can fire before one exists, and an empty identifier reads as a defect rather than as an absence. ## What it cost Nothing new is plumbed and no identifier is invented. The value is the turn's own trace, already carried by every metadata log inside it, so an operator pastes it into SigNoz and has the whole turn. ## One thing worth a look, not blocking A trace ID is not member data and is inert on its own, but it is an internal identifier appearing in a public channel. I treated that as intended because it is the point of the feature. If it should appear only in `#bots` and not on the HTTP profile, that is a one-line condition. `ward exec gate` green: build, policy-check, vet, test, test-skips, pre-commit.

coilyco-ops added 2 commits

2026-08-13 10:18:58 +00:00

fix(proxy): say how much reasoning a spent budget bought

ci / test (pull_request) Successful in 32s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

ci / image-build (pull_request) Successful in 20s

Details

7a4ea80f02

A knowledge question returned empty content 4 of 10 times after burning 3600
tokens and two raises. Every failure rendered identically, so two different
conditions were indistinguishable:

  the model thought and ran out    reasoning content large, budget too small
  the model produced nothing       reasoning content empty, not a budget problem

Those want opposite responses. One is a ceiling to raise, the other is a
question about the model, and the message could not tell them apart.

chatResponseMessage already parsed reasoning_content and nothing read it. The
failure and model.budget.raised now carry its size, so the next run says which
failure it was with a number rather than leaving it to be re-measured.

A byte count, never the text. Reasoning content is model output and the turn
logger carries no model bodies, so a test asserts the message leaks none of it.

The escalation itself is not changed and is not broken. 900 to 1800 to 3600
across two raises is what its documentation says, and truncated() correctly
requires a length finish with empty content and no tool calls, so a length
finish that still produced an answer stays usable. Tests pin all three.

Deliberately not touching maxCompletionTokens, budgetRaisesAllowed, or
completionBudgetStep. Whether 3600 is enough for a reasoning model answering a
knowledge question is a cost and latency decision against a live route, and
raising a ceiling because a measurement was uncomfortable is how a budget stops
meaning anything.

closes #325

Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

feat(notice): cite the turn's trace when it ends in anything but success

ci / test (pull_request) Successful in 30s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

ci / image-build (pull_request) Successful in 18s

Details

26c606c952

A member reporting "it said no and I do not know why" was handing over a
report. Now they hand over a query.

  > `turn timed out, retry shortly`
  > `trace id 3dd883c6becba130e9f8b75e4593a94d`

Covers stage failures, timeouts, rate-limit cooldowns, queue sheds, and an
undelivered reply. A successful reply never gains one, because the line is what
marks a turn that did not succeed.

Two deviations from the request, both deliberate.

It reads trace id rather than trace ID: because both lines take the notice
alphabet, which admits [a-z0-9 ,./-] and nothing else. A colon cannot survive
it. Widening the alphabet for one label would weaken the property that makes a
notice recognisable at a glance, which is worth more than the punctuation.

Outside a span the line is omitted rather than rendered blank. Not every
refusal happens inside a turn span, a rate-limit shed can fire before one
exists, and an empty identifier reads as a defect rather than an absence.

No new identifier and nothing new plumbed. The value is the turn's own trace,
already carried by every metadata log inside it, so an operator pastes it into
SigNoz and has the whole turn.

Tests pin all four properties, including that both lines still match the
notice shape, since a notice a member cannot distinguish from model output has
lost the thing it exists for.

closes #336

Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

coilyco-ops requested review from coilysiren

2026-08-13 10:18:58 +00:00

coilyco-ops referenced this pull request

2026-08-13 10:21:11 +00:00

Have the harness append a trace ID to every message that ends in anything other than success #336