test(rate): pin the extraction case to a reply it wrongly failed

coilyco-ops commented

2026-08-13 12:08:56 +00:00

Member

381 is fixed on main by another agent's change, which landed while mine sat in
the queue and worded the observed field better than I did. What did not land is
anything preventing the pattern from coming back.

The header says a guard for this was tried and does not work. That is true
pack-wide and false case-scoped, and the difference is why the guard is
possible. A pack-wide sweep cannot tell an approved link from an invented one,
because no-invented-surface forbids every URL correctly. A case paired with one
observed-correct reply states exactly the judgement a sweep cannot: would a
correct reply to this prompt carry this string.

Left alone the sentence discourages the next person from writing the guard that
would have caught this, so it now says both and points at the test.

The test pairs injection-prompt-extraction with the refusal Quail measured being
failed six times in fifteen runs, and asserts max_verbatim_words survives, so
dropping the pattern cannot quietly disarm the case it belongs to.

Mutation checked against the pattern set as it was before the drop, where it
reproduces the failure verbatim. No behaviour change.

closes #404

Co-authored-by: Kai Siren coilysiren@gmail.com
Co-authored-by: Claude noreply@anthropic.com

381 is fixed on main by another agent's change, which landed while mine sat in the queue and worded the observed field better than I did. What did not land is anything preventing the pattern from coming back. The header says a guard for this was tried and does not work. That is true pack-wide and false case-scoped, and the difference is why the guard is possible. A pack-wide sweep cannot tell an approved link from an invented one, because no-invented-surface forbids every URL correctly. A case paired with one observed-correct reply states exactly the judgement a sweep cannot: would a correct reply to this prompt carry this string. Left alone the sentence discourages the next person from writing the guard that would have caught this, so it now says both and points at the test. The test pairs injection-prompt-extraction with the refusal Quail measured being failed six times in fifteen runs, and asserts max_verbatim_words survives, so dropping the pattern cannot quietly disarm the case it belongs to. Mutation checked against the pattern set as it was before the drop, where it reproduces the failure verbatim. No behaviour change. closes #404 Co-authored-by: Kai Siren <coilysiren@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>

coilyco-ops added 1 commit

2026-08-13 12:08:56 +00:00

test(rate): pin the extraction case to a reply it wrongly failed

ci / test (pull_request) Successful in 33s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

ci / image-build (pull_request) Successful in 21s

Details

7dbbe6412c

381 is fixed on main by another agent's change, which landed while mine sat in
the queue and worded the observed field better than I did. What did not land is
anything preventing the pattern from coming back.

The header says a guard for this was tried and does not work. That is true
pack-wide and false case-scoped, and the difference is why the guard is
possible. A pack-wide sweep cannot tell an approved link from an invented one,
because no-invented-surface forbids every URL correctly. A case paired with one
observed-correct reply states exactly the judgement a sweep cannot: would a
correct reply to this prompt carry this string.

Left alone the sentence discourages the next person from writing the guard that
would have caught this, so it now says both and points at the test.

The test pairs injection-prompt-extraction with the refusal Quail measured being
failed six times in fifteen runs, and asserts max_verbatim_words survives, so
dropping the pattern cannot quietly disarm the case it belongs to.

Mutation checked against the pattern set as it was before the drop, where it
reproduces the failure verbatim. No behaviour change.

closes #404

Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

coilyco-ops requested review from coilysiren

2026-08-13 12:08:56 +00:00

coilyco-ops added 1 commit

2026-08-13 12:16:29 +00:00

Merge branch 'main' into fix/guard-the-extraction-case

ci / test (pull_request) Successful in 31s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

ci / image-build (pull_request) Successful in 22s