fix(coalesce): tell the model a folded ask is several comments #983

Merged

coilysiren merged 1 commit from aos/claude/folded-ask-coverage into main

2026-08-18 21:09:30 +00:00

coilyco-ops commented

2026-08-18 21:01:49 +00:00

Member

A gap in #975, found by Kai asking whether a small model would fall over on a folded ask. It would, and this is why.

The defect

A batch reached the model as its comments concatenated with a blank line between them and nothing anywhere saying the ask was multi-part. The bridge has Batch.Criteria() for exactly this ("Cover every comment below and nothing beyond them"), and I deliberately did not use it on the Discord path, because it renders surface, channel, and author into the prompt and the identifier guard refuses a reply carrying any value this process holds. Dropping the framing along with the identifiers was the mistake.

A strong model infers coverage. A weaker one answers the last comment and drops the ones before it. Those dropped comments already carry the arrival mark that promised an answer, so the failure is silent and lands in front of a member.

The fix

TranscriptEntry.Comments, rendered by foldedSuffix() beside the suffixes that already report an agent author, an attachment, and an asserted entry. It goes in the turn context, not in the member's content, so nothing harness-authored is scored as something a member said.

The request that follows is from ana. It carries 3 comments from that person,
and answering it means answering every one of them.

A count and never an identifier. A test asserts the rendered context carries no transport identifier, so this cannot quietly grow into the thing it was avoiding.

An unfolded turn sets Comments: 1 and renders nothing, so every path that is not the coalescing lane is unchanged.

The rate case, which is NOT MEASURED

coalesced-ask-answers-every-comment in agents/deep/packs/rate.yaml: three sums in one ask, comments: 3, and a required pattern per answer. A dropped comment is a missing number.

It has never been run. Its observed line says exactly that rather than carrying a number I did not measure. max_failure_rate: 0.0 is a target, not a result. The pack gates nothing, so this cannot read as a passing check anywhere.

Its language scope is recorded as any-language: the answers are numerals, so a dropped comment is a missing number whatever language the reply is written in.

Why it matters now

Deep and dowel take the coalescing lane in coilyco-bridge/deploy#682, both on DeepSeek, and dowel is the lane being recorded. Echo is held off that lane on Kai's call, and echo is the one lane whose model is not visible in the deploy manifests.

The honest read: this makes a folded ask much more likely to be answered completely and does not prove it. What would prove it is running just rate-deep against the new case and replacing that observed line with a number.

Verification

just gate passes: build, policy-check, vet, test, test-skips, pre-commit. go test -race ./... clean. Rendered prompt snapshots unchanged, since this touches turn context rather than the system prompt.

Three new tests: the folded ask carries the count and the coverage sentence and no identifier, an unfolded ask says nothing about coverage at 0 and at 1 comment, and the Discord turn reports its own count.

🤖 Generated with Claude Code

A gap in #975, found by Kai asking whether a small model would fall over on a folded ask. It would, and this is why. ## The defect A batch reached the model as its comments concatenated with a blank line between them and **nothing anywhere saying the ask was multi-part**. The bridge has `Batch.Criteria()` for exactly this ("Cover every comment below and nothing beyond them"), and I deliberately did not use it on the Discord path, because it renders surface, channel, and author into the prompt and the identifier guard refuses a reply carrying any value this process holds. Dropping the framing along with the identifiers was the mistake. A strong model infers coverage. A weaker one answers the last comment and drops the ones before it. **Those dropped comments already carry the arrival mark that promised an answer**, so the failure is silent and lands in front of a member. ## The fix `TranscriptEntry.Comments`, rendered by `foldedSuffix()` beside the suffixes that already report an agent author, an attachment, and an asserted entry. It goes in the turn **context**, not in the member's content, so nothing harness-authored is scored as something a member said. ``` The request that follows is from ana. It carries 3 comments from that person, and answering it means answering every one of them. ``` **A count and never an identifier.** A test asserts the rendered context carries no transport identifier, so this cannot quietly grow into the thing it was avoiding. An unfolded turn sets `Comments: 1` and renders nothing, so every path that is not the coalescing lane is unchanged. ## The rate case, which is NOT MEASURED `coalesced-ask-answers-every-comment` in `agents/deep/packs/rate.yaml`: three sums in one ask, `comments: 3`, and a required pattern per answer. A dropped comment is a missing number. **It has never been run.** Its `observed` line says exactly that rather than carrying a number I did not measure. `max_failure_rate: 0.0` is a target, not a result. The pack gates nothing, so this cannot read as a passing check anywhere. Its language scope is recorded as `any-language`: the answers are numerals, so a dropped comment is a missing number whatever language the reply is written in. ## Why it matters now Deep and dowel take the coalescing lane in coilyco-bridge/deploy#682, both on DeepSeek, and dowel is the lane being recorded. **Echo is held off that lane** on Kai's call, and echo is the one lane whose model is not visible in the deploy manifests. The honest read: this makes a folded ask much more likely to be answered completely and **does not prove it**. What would prove it is running `just rate-deep` against the new case and replacing that `observed` line with a number. ## Verification `just gate` passes: build, policy-check, vet, test, test-skips, pre-commit. `go test -race ./...` clean. Rendered prompt snapshots unchanged, since this touches turn context rather than the system prompt. Three new tests: the folded ask carries the count and the coverage sentence and no identifier, an unfolded ask says nothing about coverage at 0 and at 1 comment, and the Discord turn reports its own count. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

coilyco-ops added 1 commit

2026-08-18 21:01:49 +00:00

fix(coalesce): tell the model a folded ask is several comments

ci / image-build (pull_request) Successful in 41s

Details

ci / test (pull_request) Successful in 1m9s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

0e6bfb190f

A batch reached the model as its comments concatenated with a blank line
between them and nothing saying the ask was multi-part. A strong model
infers coverage. A weaker one answers the last comment and drops the ones
before it, and those already carry the arrival mark that promised an
answer, so the failure is silent and lands in front of a member.

The turn context now carries the count and says answering the request
means answering every comment in it, beside the suffixes that already
report an agent author, an attachment, and an asserted entry. A count and
never a channel, author, or surface, because the identifier guard refuses
a reply carrying any value this process holds.

The rate case is added and NOT MEASURED. Its observed line says so.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

coilyco-ops requested review from coilysiren

2026-08-18 21:01:49 +00:00