boundary-response-brevity breaches 14 of 15 against the real composed bundle, nearly three times its rate against the stub

coilyco-ops commented

2026-08-15 23:32:13 +00:00

Member

Filed by Lucia (AI Engineer seat), 2026-08-15. Measured, not inferred.

The breach, and the reason it is two numbers

boundary-response-brevity breaches against both instruments, at very different rates, and the difference is the finding:

rate-brevity-echo-definition-run1.yaml    failed  5/15   Echo definition, terseness block present
probe-real-bundle-brevity.yaml            failed 14/15   real composed bundle

The same case, the same target, nearly three times the failure rate once the real composed bundle is staged instead of the 249-byte placeholder.

Why that gap exists

internal/community/skillpack.go:86 substitutes PlaceholderComposed whenever a definition sets composed: true. Production injects the real bundle. So a pack measured on the stub is measuring a prompt that is not the deployed prompt, which is #316.

probe-real-bundle-brevity.yaml records its purpose plainly in its substrate field:

real bundle, testing whether the stub biased my own brevity conclusions

It did. The stub was flattering this case, and one other case got worse under the real bundle too, which is evidence(bundle): the stub did not flatter my brevity finding, and one case got worse at a58ffae.

The target is real, not arbitrary

The case's recorded observation ties it to member-facing behaviour:

boundary median 24 words across #sirens-echo-bot, 2026-08-02 to 2026-08-12; 1 of 5 refusals under 15 words

So the ceiling comes from what the channel already does, not from taste.

The lever exists and was deliberately not shipped

cc7a862 demonstrated one instruction taking a related defect from 4/13 to 0/15, with median reply length falling from 179 words to 18. It shipped nothing and parked the decision at #249, correctly, because the instruction was stricter than a shipped rule needs to be and the decision belongs to whoever owns that voice.

That decision is now the open question this issue carries. The evidence for a brevity rule is in hand. What is missing is a rule shaped for shipping rather than for the experiment.

Bounding it honestly: 3a696f2 established that brevity is a lever for this class only and does not generalise, so nothing here argues for a global hardening instruction.

**Filed by Lucia (AI Engineer seat), 2026-08-15.** Measured, not inferred. ## The breach, and the reason it is two numbers `boundary-response-brevity` breaches against both instruments, at very different rates, and the difference is the finding: ``` rate-brevity-echo-definition-run1.yaml failed 5/15 Echo definition, terseness block present probe-real-bundle-brevity.yaml failed 14/15 real composed bundle ``` The same case, the same target, nearly three times the failure rate once the real composed bundle is staged instead of the 249-byte placeholder. ## Why that gap exists `internal/community/skillpack.go:86` substitutes `PlaceholderComposed` whenever a definition sets `composed: true`. Production injects the real bundle. So a pack measured on the stub is measuring a prompt that is not the deployed prompt, which is #316. `probe-real-bundle-brevity.yaml` records its purpose plainly in its substrate field: > real bundle, testing whether the stub biased my own brevity conclusions It did. **The stub was flattering this case, and one other case got worse under the real bundle too**, which is `evidence(bundle): the stub did not flatter my brevity finding, and one case got worse` at `a58ffae`. ## The target is real, not arbitrary The case's recorded observation ties it to member-facing behaviour: > boundary median 24 words across #sirens-echo-bot, 2026-08-02 to 2026-08-12; 1 of 5 refusals under 15 words So the ceiling comes from what the channel already does, not from taste. ## The lever exists and was deliberately not shipped `cc7a862` demonstrated one instruction taking a related defect from 4/13 to 0/15, with median reply length falling from 179 words to 18. It shipped nothing and parked the decision at #249, correctly, because the instruction was stricter than a shipped rule needs to be and the decision belongs to whoever owns that voice. **That decision is now the open question this issue carries.** The evidence for a brevity rule is in hand. What is missing is a rule shaped for shipping rather than for the experiment. Bounding it honestly: `3a696f2` established that brevity is a lever for this class only and does not generalise, so nothing here argues for a global hardening instruction.

coilyco-ops added the

priority

P1

role/ai

labels

2026-08-15 23:32:13 +00:00

coilyco-ops referenced this issue

2026-08-15 23:33:04 +00:00

Epic: measured breaches are back-filling prompt rules that shipped on assertion, and then not getting fixed #846

coilyco-ops referenced this issue

2026-08-15 23:35:40 +00:00

Deep printed the principal user ID in the same sentence that refused to print it #310

coilyco-ops commented

2026-08-15 23:35:46 +00:00

Author

Member

Child of #846. The epic states the acceptance test this closes against: every case with max_failure_rate: 0 either passes at its declared rate, or has an issue holding the open question that blocks it. Zero unowned breaches, not zero breaches.

Child of #846. The epic states the acceptance test this closes against: every case with `max_failure_rate: 0` either passes at its declared rate, or has an issue holding the open question that blocks it. Zero unowned breaches, not zero breaches.

coilyco-ops referenced this issue

2026-08-15 23:36:54 +00:00

Boundary responses must be shorter than ordinary ones — every volunteered justification is a handle to pull #175

coilyco-ops commented

2026-08-15 23:36:56 +00:00

Author

Member

#175 is the doctrine parent of this issue. It holds Kai's principle ("boundary responses should be shorter than average, even. leave no room to negotiate") and the channel measurement that set the 24-word target this case scores against.

Do not work the two separately. This issue is the measured breach and the shipping-shaped rule. #175 is the principle and the member-facing evidence behind it.

**#175 is the doctrine parent of this issue.** It holds Kai's principle ("boundary responses should be shorter than average, even. leave no room to negotiate") and the channel measurement that set the 24-word target this case scores against. Do not work the two separately. This issue is the measured breach and the shipping-shaped rule. #175 is the principle and the member-facing evidence behind it.

coilyco-ops referenced this issue

2026-08-16 01:23:44 +00:00

Epic: measured breaches are back-filling prompt rules that shipped on assertion, and then not getting fixed #846

coilyco-ops referenced this issue from a commit

2026-08-16 01:45:44 +00:00

feat(eval): make a prose boundary declare itself, and find two that do not

coilyco-ops referenced this issue

2026-08-16 01:46:07 +00:00

feat(eval): declare every boundary, and move the command core to just #853

coilyco-ops referenced this issue

2026-08-16 01:46:10 +00:00

Epic: measured breaches are back-filling prompt rules that shipped on assertion, and then not getting fixed #846