fix: ship a well-formed reply a quality rule refused

coilyco-ops commented

2026-08-17 18:08:38 +00:00

Member

Implements Kai's decision: a well-formed reply the model finished normally is no longer discarded by a validator rejection.

The split, which is the reviewable part

Checks divide on what the reply would cost a member, not on which rule fired.

Quality, records and ships: response_style, tool_call_markup. Both are about how an answer reads.
Gates: everything else. parse, the four grounding rules, self_attributed_claim, identifier_disclosure, identity_claim.

checkGates reads an allowlist rather than a blocklist, so a rule nobody classified gates. A new check has to be argued into the quality tier rather than falling into it.

The two judgement calls, stated so they are easy to disagree with:

The grounding family gates. A reply asserting the service filed an issue it did not file is a false claim about an irreversible action, which is #179's second tier. That is not mild wrongness: the member stops filing it themselves.
self_attributed_claim gates for the same reason.

Moving a check between tiers is one line in qualityChecks.

Both layers

Proxy. The repair loop still repairs once, then ships instead of returning ErrResponseRepairExhausted when the outstanding refusal is quality. This is the path the issue's trace took.
Agent. The validation stage annotates and delivers, after the existing redaction rung has had its go, so a droppable block is still dropped rather than shipped.

Telemetry

model.response.shipped at the proxy, response.check.shipped on the response.validate span and log, both carrying the rule and the reason. A gating refusal is byte-identical: same notice, same failure_cause, same exception fields.

Tests

replyshipping_test.go - a quality refusal reaches the member and names its rule, a blast-radius refusal does not, and the fail-closed default is asserted over every check including an invented future one.
repairnotice_test.go - the four criterion-1 tests keep every assertion and move to a gating fixture, since a style refusal no longer exhausts repair. Added TestTheAnswerFromTheTraceNowReachesTheMember: the verbatim answer from the trace is returned rather than discarded.
TestProxyClientRejectsPersistentStyleViolation renamed to ...Ships... and inverted. That is the behaviour the decision reverses.
TestAGenuineBackendFailureStillReportsAnOutage untouched and passing.

One thing worth reviewing

docs/sirens-echo-delivery.md was at its 8,000-char cap, so four restatements elsewhere on the page were compressed to make room. No live fact was dropped, but the diff touches prose this change is not about.

What #675 still owns

The false model backend unavailable for the paths that do still fail. Unchanged here.

closes #651

Implements Kai's decision: a well-formed reply the model finished normally is no longer discarded by a validator rejection. ## The split, which is the reviewable part Checks divide on **what the reply would cost a member**, not on which rule fired. * **Quality, records and ships:** `response_style`, `tool_call_markup`. Both are about how an answer reads. * **Gates:** everything else. `parse`, the four `grounding` rules, `self_attributed_claim`, `identifier_disclosure`, `identity_claim`. `checkGates` reads an allowlist rather than a blocklist, **so a rule nobody classified gates**. A new check has to be argued into the quality tier rather than falling into it. The two judgement calls, stated so they are easy to disagree with: * **The grounding family gates.** A reply asserting the service filed an issue it did not file is a false claim about an irreversible action, which is #179's second tier. That is not mild wrongness: the member stops filing it themselves. * **`self_attributed_claim` gates** for the same reason. Moving a check between tiers is one line in `qualityChecks`. ## Both layers * **Proxy.** The repair loop still repairs once, then ships instead of returning `ErrResponseRepairExhausted` when the outstanding refusal is quality. This is the path the issue's trace took. * **Agent.** The validation stage annotates and delivers, after the existing redaction rung has had its go, so a droppable block is still dropped rather than shipped. ## Telemetry `model.response.shipped` at the proxy, `response.check.shipped` on the `response.validate` span and log, both carrying the rule and the reason. A gating refusal is byte-identical: same notice, same `failure_cause`, same exception fields. ## Tests * `replyshipping_test.go` - a quality refusal reaches the member and names its rule, a blast-radius refusal does not, and the fail-closed default is asserted over every check including an invented future one. * `repairnotice_test.go` - the four criterion-1 tests keep every assertion and move to a gating fixture, since a style refusal no longer exhausts repair. Added `TestTheAnswerFromTheTraceNowReachesTheMember`: the verbatim answer from the trace is returned rather than discarded. * `TestProxyClientRejectsPersistentStyleViolation` renamed to `...Ships...` and inverted. That is the behaviour the decision reverses. * `TestAGenuineBackendFailureStillReportsAnOutage` untouched and passing. ## One thing worth reviewing `docs/sirens-echo-delivery.md` was at its 8,000-char cap, so four restatements elsewhere on the page were compressed to make room. No live fact was dropped, but the diff touches prose this change is not about. ## What #675 still owns The false `model backend unavailable` for the paths that do still fail. Unchanged here. closes #651

coilyco-ops added 1 commit

2026-08-17 18:08:39 +00:00

fix: ship a well-formed reply a quality rule refused

ci / image-build (pull_request) Successful in 32s

Details

ci / test (pull_request) Successful in 58s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

c8aebfb3c5

A member asked a 53-byte question, the model answered correctly twice, and
both answers were discarded for first-person voice under the neutral profile.
The member was told the backend was down while it had answered 200 in under
eight seconds. Per Kai's decision on the issue, a validator rejection no
longer discards a well-formed reply.

Reply checks split by what the reply would cost a member rather than by which
rule fired. A quality rule is about how an answer reads: response_style and
tool_call_markup record the refusal and the reply still ships. Everything else
gates, and checkGates reads an allowlist so a rule nobody classified fails
closed rather than shipping.

Both layers. The proxy's repair loop still repairs once and then ships instead
of returning ErrResponseRepairExhausted for a quality rule, which is the path
this issue's trace took. The agent's validation stage annotates and delivers,
after the existing redaction rung has had its go.

New records: model.response.shipped at the proxy and response.check.shipped on
the response.validate span and log, both carrying the rule and the reason. A
gating refusal is untouched: same notice, same failure cause, same exception
fields.

docs/sirens-echo-delivery.md was at its character cap, so four restatements
elsewhere on the page were compressed to make room. No live fact was dropped.

closes #651

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

coilyco-ops requested review from coilysiren

2026-08-17 18:08:39 +00:00

coilyco-ops referenced this pull request

2026-08-17 18:09:01 +00:00

The repair path discarded two correct answers and told the member the backend was down #651

coilyco-ops added 1 commit

2026-08-17 20:19:09 +00:00

Merge main: compose the silence guard with the check tiers

ci / image-build (pull_request) Successful in 30s

Details

ci / test (pull_request) Successful in 58s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

c1ce5ad737

main gained the silent turn from #895 while this branch was open, and both
sides edit the same block of Complete. The conflict was one hunk and both
edits belong.

unchosenSilence keeps its guard, and it reports under replyCheckParse, which
gates. A turn that ran no tool and returned nothing has nothing to ship, so it
must not reach the quality-rule path this branch adds.

Chosen silence is unaffected: tools ran, unchosenSilence is false, the checks
pass on empty, and finishSilently posts nothing. Both sides' tests run together
and pass, which is the part that matters here.

Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>