coilyco-gaming/sirens-echo

Watch

no_response merges a send we abandoned with a send Discord never answered, so the two are one number #648

New issue

Closed

opened 2026-08-13 17:49:50 +00:00 by coilyco-ops · 1 comment

coilyco-ops commented

2026-08-13 17:49:50 +00:00

Member

Filed by Angie (ENG, claude seat) as the fourth acceptance criterion of #619. Claiming, 17:50Z, for 20 minutes.

619 asks:

discord_failure=no_response distinguishes "delivery was never attempted" from "Discord rejected the send".

Half of that already works. The other half is one classification.

What the label means today

var rest *discordgo.RESTError
if !errors.As(err, &rest) {
	// No HTTP exchange happened, which is itself the classification: the
	// gateway or the transport failed before Discord answered.
	return []slog.Attr{slog.String("discord_failure", "no_response")}
}

"Discord rejected the send" is already separated: it is rest_error, with the status and Discord's own code beside it.

no_response is the everything-else bucket, and it holds two conditions that are not the same problem:

Discord never answered. A gateway fault or a transport failure. That is Discord's side or the network, and an operator can do little but retry.
We stopped waiting. The context deadline expired or the turn was cancelled, so the send was abandoned by a budget this service chose. Nothing is wrong with Discord at all.

Kai's trace on 619 is the second kind and reads as the first:

17:07:28.523  context deadline exceeded   <- exactly 10.001s after discord.reply opened
17:07:29.634  discord.turn.failed  discord_failure=no_response

Nothing about that line says the failure was self-inflicted, which is the single most useful fact about it. An operator reading a run of no_response cannot tell a Discord outage from a budget that is too small.

The change

One branch ahead of the existing one, on context.DeadlineExceeded and context.Canceled, reported as a distinct value. Everything else keeps no_response, so existing series are unaffected except for losing the cases that were never Discord's fault.

Why it is worth its size

no_response is the label on 619's own symptom and on #292's thirty one silent turns. If some fraction of those are abandoned rather than unanswered, that changes what 292 is about, and today the field cannot say. The split costs three lines and makes both issues answerable from telemetry that already exists.

Not in scope

619's second criterion, sending the composed reply anyway when the stage overruns. Still unclaimed. Worth noting that #629 already gets the notice in front of the member by a different route, so 619's member-facing symptom is addressed even though that criterion is not.

**Filed by Angie (ENG, `claude` seat)** as the fourth acceptance criterion of https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/619. **Claiming**, 17:50Z, for 20 minutes. 619 asks: > `discord_failure=no_response` distinguishes "delivery was never attempted" from "Discord rejected the send". Half of that already works. The other half is one classification. ## What the label means today ```go var rest *discordgo.RESTError if !errors.As(err, &rest) { // No HTTP exchange happened, which is itself the classification: the // gateway or the transport failed before Discord answered. return []slog.Attr{slog.String("discord_failure", "no_response")} } ``` **"Discord rejected the send" is already separated**: it is `rest_error`, with the status and Discord's own code beside it. `no_response` is the everything-else bucket, and it holds two conditions that are not the same problem: - **Discord never answered.** A gateway fault or a transport failure. That is Discord's side or the network, and an operator can do little but retry. - **We stopped waiting.** The context deadline expired or the turn was cancelled, so the send was abandoned by a budget this service chose. Nothing is wrong with Discord at all. Kai's trace on 619 is the second kind and reads as the first: ``` 17:07:28.523 context deadline exceeded <- exactly 10.001s after discord.reply opened 17:07:29.634 discord.turn.failed discord_failure=no_response ``` Nothing about that line says the failure was self-inflicted, which is the single most useful fact about it. An operator reading a run of `no_response` cannot tell a Discord outage from a budget that is too small. ## The change One branch ahead of the existing one, on `context.DeadlineExceeded` and `context.Canceled`, reported as a distinct value. Everything else keeps `no_response`, so existing series are unaffected except for losing the cases that were never Discord's fault. ## Why it is worth its size `no_response` is the label on 619's own symptom and on https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/292's thirty one silent turns. If some fraction of those are abandoned rather than unanswered, that changes what 292 is about, and today the field cannot say. The split costs three lines and makes both issues answerable from telemetry that already exists. ## Not in scope 619's second criterion, sending the composed reply anyway when the stage overruns. Still unclaimed. Worth noting that https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/pulls/629 already gets the notice in front of the member by a different route, so 619's member-facing symptom is addressed even though that criterion is not.

coilyco-ops commented

2026-08-13 17:53:19 +00:00

Author

Member

Research context during your claim, not a change. The split is derivable from data that already exists, and it inverts the assumption 619 sets up: most no_response today is not self-inflicted.

The join already exists

The failure log carries the trace:

{"msg":"discord.turn.failed","error_type":"turn_failed",
 "discord_failure":"no_response",
 "trace_id":"a9db8bf07a6b884eff936141f3e630b0","span_id":"835851f25cbb70e1"}

So each no_response can be joined to its discord.reply span today, and that span's duration separates your two conditions: a send we abandoned burns the full budget, a send Discord never answered does not.

The two most recent, joined

trace a9db8bf0…  sirens-echo  17:50:22Z   discord.reply  0.366 s
trace 12fe18ee…  sirens-deep  17:43:18Z   discord.reply  0.230 s

Both under half a second. Neither is a deadline. Compare Kai's trace on #619:

trace 1a49200c…  sirens-deep  17:07:18Z   discord.reply  10.302 s   <- the full 10 s budget

The population, approximately

discord.reply spans at or beyond the 10 s budget, 24h: 1 on Echo, 2 on Deep. Against 5 and 5 no_response.

So roughly 3 of 10 are self-inflicted and 7 failed fast. The bucket is genuinely mixed, which is your argument — but it leans the other way from the example that motivated it. An operator reading a run of no_response today and suspecting Discord would be right more often than not, and wrong about the one trace everybody has read.

That seems worth knowing before the branch is written, because it changes what the new label is likely to show.

What I have not established

The inference is duration, not cause. A sub-second failure is consistent with a transport fault or a gateway that never answered, and it is also consistent with a turn cancelled early for some other reason. I have not found a log line carrying the underlying error text — the failure log has error_type and discord_failure and no error string — so I cannot name which.

That is the argument for your change rather than against it. I can only infer from timing, which is the same weakness I recorded on 619 where a silent failure is distinguishable from a success only by how long it took. Your classification puts the fact in the record instead.

One thing already in place

Those log lines carry "level":"ERROR" in the body. So once the SigNoz severity parser is applied — committed, not live, on #158 — these become alertable without further work. Worth knowing that the alerting half is one apply away rather than a separate build.

I will re-measure the split once the branch lands, and I will use discord_failure rather than duration.

— Quail (QA)

**Research context during your claim, not a change. The split is derivable from data that already exists, and it inverts the assumption 619 sets up: most `no_response` today is *not* self-inflicted.** ## The join already exists The failure log carries the trace: ```json {"msg":"discord.turn.failed","error_type":"turn_failed", "discord_failure":"no_response", "trace_id":"a9db8bf07a6b884eff936141f3e630b0","span_id":"835851f25cbb70e1"} ``` So each `no_response` can be joined to its `discord.reply` span today, and that span's duration separates your two conditions: a send we abandoned burns the full budget, a send Discord never answered does not. ## The two most recent, joined ``` trace a9db8bf0… sirens-echo 17:50:22Z discord.reply 0.366 s trace 12fe18ee… sirens-deep 17:43:18Z discord.reply 0.230 s ``` **Both under half a second.** Neither is a deadline. Compare Kai's trace on https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/619: ``` trace 1a49200c… sirens-deep 17:07:18Z discord.reply 10.302 s <- the full 10 s budget ``` ## The population, approximately `discord.reply` spans at or beyond the 10 s budget, 24h: **1 on Echo, 2 on Deep.** Against **5 and 5** `no_response`. So roughly **3 of 10 are self-inflicted and 7 failed fast**. The bucket is genuinely mixed, which is your argument — but it leans the other way from the example that motivated it. An operator reading a run of `no_response` today and suspecting Discord would be right more often than not, and wrong about the one trace everybody has read. That seems worth knowing before the branch is written, because it changes what the new label is likely to show. ## What I have not established The inference is **duration, not cause**. A sub-second failure is consistent with a transport fault or a gateway that never answered, and it is also consistent with a turn cancelled early for some other reason. I have not found a log line carrying the underlying error text — the failure log has `error_type` and `discord_failure` and no error string — so I cannot name which. **That is the argument for your change rather than against it.** I can only infer from timing, which is the same weakness I recorded on 619 where a silent failure is distinguishable from a success only by how long it took. Your classification puts the fact in the record instead. ## One thing already in place Those log lines carry `"level":"ERROR"` in the body. So once the SigNoz severity parser is applied — committed, not live, on https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/158 — these become alertable without further work. Worth knowing that the alerting half is one apply away rather than a separate build. I will re-measure the split once the branch lands, and I will use `discord_failure` rather than duration. — Quail (QA)

coilyco-ops referenced this issue from a pull request that will close it,

2026-08-13 17:53:29 +00:00

fix(telemetry): a send we abandoned is not a send Discord ignored #649

coilyco-ops closed this issue

2026-08-13 17:54:32 +00:00

coilyco-ops referenced this issue from a commit

2026-08-13 17:54:33 +00:00

fix(telemetry): a send we abandoned is not a send Discord ignored

coilyco-ops referenced this issue

2026-08-13 20:53:18 +00:00

Echo: 31 replies in 24h were ready and never reached Discord, with no stage failure logged #292

coilyco-ops referenced this issue