coilyco-gaming/sirens-echo

Watch

Have the harness append a trace ID to every message that ends in anything other than success #336

New issue

Closed

opened 2026-08-13 10:11:20 +00:00 by coilysiren · 2 comments

coilysiren commented

2026-08-13 10:11:20 +00:00

Owner

So: classifier blocks, rate limits, service downtime - all should emit a trace ID

I think most it not all of those are using this format

500 server error

so you would just append

500 server error
trace ID: 1234

So: classifier blocks, rate limits, service downtime - all should emit a trace ID I think most it not all of those are using this format > `500 server error` so you would just append > `500 server error` > `trace ID: 1234`

coilyco-ops commented

2026-08-13 10:12:00 +00:00

Member

CLAIM — Angie (ENG, claude seat) · 2026-08-13T10:13Z · 20 min.

Taking it. This is a good ask and it lands on a surface that already exists, so it should be small.

What I read the ask as

Every harness-generated message that is not a successful reply carries the trace ID of the turn that produced it, appended in the shape you gave:

> `turn timed out, retry shortly`
> `trace ID: 3dd883c6becba130e9f8b75e4593a94d`

That covers classifier blocks, rate-limit sheds, stage failures, and the service-unavailable notices. If a member reports "it said no and I do not know why", the trace ID is the thing that turns their screenshot into a query.

Why it is cheap here

Trace correlation already exists. Every metadata log inside a trace carries trace_id and span_id, and Telemetry.log pulls it from the span context. So the value is in hand at the moment the notice is built. Nothing new has to be plumbed, and no new identifier is invented.

Two things I will check rather than assume

That every non-success path actually goes through one notice constructor. The repo has feat(notice): give harness-generated messages one strict shape, so I expect one place. If some paths bypass it, the fix is routing them through it rather than appending in five places, and I will say so if that turns out to be the shape.

That a notice emitted outside a span does something sensible. Not every refusal happens inside a turn span. A rate-limit shed can fire before the turn starts, and a trace ID: line reading empty would be worse than no line. It will omit the line rather than print a blank one, and I will test that case explicitly.

One thing worth your view, though I am not blocking on it

A trace ID is not member data, but it is an internal identifier appearing in a public channel. It is inert on its own — it identifies a span in SigNoz that only an operator can read. I am treating that as fine because you asked for it and because it is the whole point of the feature, but flagging that I noticed rather than deciding it silently. If you would rather it appear only in #bots and not on the HTTP profile, say so and it is a one-line condition.

Reporting what lands and what it does not cover.

**CLAIM — Angie (ENG, claude seat)** · 2026-08-13T10:13Z · 20 min. Taking it. This is a good ask and it lands on a surface that already exists, so it should be small. ## What I read the ask as Every harness-generated message that is **not** a successful reply carries the trace ID of the turn that produced it, appended in the shape you gave: ``` > `turn timed out, retry shortly` > `trace ID: 3dd883c6becba130e9f8b75e4593a94d` ``` That covers classifier blocks, rate-limit sheds, stage failures, and the service-unavailable notices. If a member reports "it said no and I do not know why", the trace ID is the thing that turns their screenshot into a query. ## Why it is cheap here Trace correlation already exists. Every metadata log inside a trace carries `trace_id` and `span_id`, and `Telemetry.log` pulls it from the span context. So the value is in hand at the moment the notice is built. Nothing new has to be plumbed, and no new identifier is invented. ## Two things I will check rather than assume **That every non-success path actually goes through one notice constructor.** The repo has `feat(notice): give harness-generated messages one strict shape`, so I expect one place. If some paths bypass it, the fix is routing them through it rather than appending in five places, and I will say so if that turns out to be the shape. **That a notice emitted outside a span does something sensible.** Not every refusal happens inside a turn span. A rate-limit shed can fire before the turn starts, and a `trace ID:` line reading empty would be worse than no line. It will omit the line rather than print a blank one, and I will test that case explicitly. ## One thing worth your view, though I am not blocking on it A trace ID is not member data, but it is an internal identifier appearing in a public channel. It is inert on its own — it identifies a span in SigNoz that only an operator can read. I am treating that as fine because you asked for it and because it is the whole point of the feature, but flagging that I noticed rather than deciding it silently. If you would rather it appear only in `#bots` and not on the HTTP profile, say so and it is a one-line condition. Reporting what lands and what it does not cover.

coilysiren referenced this issue

2026-08-13 10:16:14 +00:00

give the harness 1st class knowledge of how to query its own trace IDs #339

coilyco-ops referenced this issue from a commit

2026-08-13 10:18:39 +00:00

feat(notice): cite the turn's trace when it ends in anything but success

coilyco-ops referenced this issue from a pull request that will close it,

2026-08-13 10:18:58 +00:00

feat(notice): cite the turn's trace when it ends in anything but success #340

coilyco-ops commented

2026-08-13 10:21:11 +00:00

Member

Delivered. PR: #340 — waiting on a director merge.

A member reporting "it said no and I do not know why" was handing over a report. Now they hand over a query.

```

`turn timed out, retry shortly`
`trace id 3dd883c6becba130e9f8b75e4593a94d`
```

Covers stage failures, timeouts, rate-limit cooldowns, queue sheds, and an undelivered reply. A successful reply never gains one — the line is what marks a turn that did not succeed.

Two deviations from what you asked, both deliberate

`trace id` rather than `trace ID:`. Both lines take the notice alphabet, which admits `[a-z0-9 ,./-]` and nothing else. A colon cannot survive it. I could have widened the alphabet for the label and chose not to: the strictness is what lets a member recognise a notice at a glance as harness output rather than model output, and that is worth more than the punctuation. A test asserts both lines still match the notice shape.

Outside a span the line is omitted rather than rendered blank. Not every refusal happens inside a turn span — a rate-limit shed can fire before one exists — and an empty identifier reads as a defect rather than as an absence.

Cost

Nothing new plumbed, no identifier invented. The value is the turn's own trace, already carried by every metadata log inside that turn, so pasting it into SigNoz returns the whole turn rather than one line.

One thing for you, not blocking the merge

A trace ID is not member data and is inert on its own, but it is an internal identifier appearing in a public channel. I treated that as intended, since it is the point of the feature. If you would rather it appear only in `#bots` and not on the HTTP profile, say so — it is a one-line condition.

Delivered. PR: https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/pulls/340 — waiting on a director merge. A member reporting \"it said no and I do not know why\" was handing over a report. Now they hand over a query. \`\`\` > \`turn timed out, retry shortly\` > \`trace id 3dd883c6becba130e9f8b75e4593a94d\` \`\`\` Covers stage failures, timeouts, rate-limit cooldowns, queue sheds, and an undelivered reply. A successful reply never gains one — the line is what marks a turn that did not succeed. ## Two deviations from what you asked, both deliberate **\`trace id\` rather than \`trace ID:\`.** Both lines take the notice alphabet, which admits \`[a-z0-9 ,./-]\` and nothing else. A colon cannot survive it. I could have widened the alphabet for the label and chose not to: the strictness is what lets a member recognise a notice at a glance as harness output rather than model output, and that is worth more than the punctuation. A test asserts both lines still match the notice shape. **Outside a span the line is omitted rather than rendered blank.** Not every refusal happens inside a turn span — a rate-limit shed can fire before one exists — and an empty identifier reads as a defect rather than as an absence. ## Cost Nothing new plumbed, no identifier invented. The value is the turn's own trace, already carried by every metadata log inside that turn, so pasting it into SigNoz returns the whole turn rather than one line. ## One thing for you, not blocking the merge A trace ID is not member data and is inert on its own, but it is an internal identifier appearing in a public channel. I treated that as intended, since it is the point of the feature. If you would rather it appear only in \`#bots\` and not on the HTTP profile, say so — it is a one-line condition.

coilyco-ops referenced this issue

2026-08-13 10:22:09 +00:00

attach discord metadata to trace tags #337

coilysiren closed this issue

2026-08-13 10:22:47 +00:00