fix(evaluation): one fail block, not two #411

Merged

coilysiren merged 2 commits from fix/one-fail-block-not-two into main

2026-08-13 12:23:08 +00:00

coilyco-ops commented

2026-08-13 12:18:13 +00:00

Member

closes #407

Two seats fixed #386 and both landed. The prints sit three lines apart in the same block and both cite the same issue. Neither conflicted textually, so the merge kept both and every gate went green.

One of them is mine.

Which one survives

The one that falls back to the raw completion. Scoring can fail before it has a parsed reply — the tool-call-markup case is exactly that — and the other print emitted an empty line in precisely the case where the reply matters most.

The test is deliberately blunt

It counts the format string in the source rather than driving the runner. That is not how I would normally pin behaviour, and it is right here: the defect was two copies of a correct line. No behavioural test would call it wrong, because printing the evidence twice still prints the evidence.

Worth saying plainly

This is the third duplicate-build today and the first that put a defect in main. The others cost effort; this one cost correctness. The coordination problem is on #353.

Quail caught it from merged main rather than from a diff, which is the only place it was visible.

ward exec gate green: build, policy-check, vet, test, test-skips, pre-commit.

closes #407 Two seats fixed https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/386 and **both landed**. The prints sit three lines apart in the same block and both cite the same issue. Neither conflicted textually, so the merge kept both and every gate went green. One of them is mine. ## Which one survives The one that falls back to the raw completion. Scoring can fail **before** it has a parsed reply — the tool-call-markup case is exactly that — and the other print emitted an empty line in precisely the case where the reply matters most. ## The test is deliberately blunt It counts the format string in the source rather than driving the runner. That is not how I would normally pin behaviour, and it is right here: the defect was **two copies of a correct line**. No behavioural test would call it wrong, because printing the evidence twice still prints the evidence. ## Worth saying plainly This is the third duplicate-build today and the first that put a defect in `main`. The others cost effort; this one cost correctness. The coordination problem is on https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/353. Quail caught it from merged `main` rather than from a diff, which is the only place it was visible. `ward exec gate` green: build, policy-check, vet, test, test-skips, pre-commit.

coilyco-ops added 1 commit

2026-08-13 12:18:13 +00:00

fix(evaluation): one fail block, not two

ci / test (pull_request) Successful in 29s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

ci / image-build (pull_request) Successful in 18s

Details

237568ba46

Two seats fixed sirens-echo#386 and both landed. The prints are three lines
apart in the same branch and both cite the same issue. Neither conflicted
textually, so the merge kept both and every gate went green.

The surviving print is the one that falls back to the raw completion, because
scoring can fail before it has a parsed reply and the other printed an empty
line in exactly the case the reply matters most.

A test counts the format string in the source rather than driving the runner.
That is a blunt instrument and it is the right one here: the defect was two
copies of a correct line, which no behavioural test would call wrong, since
printing the evidence twice still prints the evidence.

closes #407

Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

coilyco-ops requested review from coilysiren

2026-08-13 12:18:13 +00:00

coilyco-ops referenced this pull request

2026-08-13 12:18:34 +00:00

Every failing battery case now prints its reply twice, because two seats fixed #386 and both landed #407

coilyco-ops added 1 commit

2026-08-13 12:20:53 +00:00

Merge branch 'main' into fix/one-fail-block-not-two

ci / test (pull_request) Successful in 32s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details