fix(evidence): preserve the longform pack, which reddened main on arrival #421

Merged

coilyco-ops merged 1 commit from qa/preserve-the-longform-pack into main

2026-08-13 12:34:19 +00:00

coilyco-ops commented

2026-08-13 12:32:50 +00:00

Member

main is red and my guard is what reddened it. This is the one-file remedy.

TestEveryCitedPackIsPreserved landed in c8e3336 (#416) and caught its first real case within minutes. 0cb3c80 added evaluations/probe-long-form-multi-turn.yaml, which cites /tmp/probe/longform.yaml — scratch space, not in the repository.

--- FAIL: TestEveryCitedPackIsPreserved
    probe-long-form-multi-turn.yaml cites /tmp/probe/longform.yaml

The pack still existed on the machine that produced it, so the evidence is intact. Copied to evaluations/packs/longform.yaml, which is exactly what the failure message asks for. Suite green.

The dataset is not edited — its pack: field still names /tmp/probe/longform.yaml, because that is what produced it.

Two things worth saying plainly

A red main blocks every open branch and skips the image publish — #305 and #307 both measured that cost. I introduced the guard, so clearing it quickly is mine, and I would rather ship this than leave it for whoever notices next.

The guard behaved as intended rather than as a nuisance, and I would say so even if it had cost more. A dataset arrived whose method was already unreachable. Without the check that becomes invisible the moment /tmp is cleaned, and the numbers in #236's evidence would join the twelve I spent this hour recovering. The gap was visible the same hour instead of next week. That is the argument for the check, made by the check.

Whether a missing pack should redden main or only warn is a fair question, and it is not mine — if the answer is warn, the test should move to a non-gating report rather than lose the coverage.

Refs #313

**`main` is red and my guard is what reddened it. This is the one-file remedy.** `TestEveryCitedPackIsPreserved` landed in `c8e3336` (#416) and caught its first real case within minutes. `0cb3c80` added `evaluations/probe-long-form-multi-turn.yaml`, which cites `/tmp/probe/longform.yaml` — scratch space, not in the repository. ``` --- FAIL: TestEveryCitedPackIsPreserved probe-long-form-multi-turn.yaml cites /tmp/probe/longform.yaml ``` **The pack still existed on the machine that produced it**, so the evidence is intact. Copied to `evaluations/packs/longform.yaml`, which is exactly what the failure message asks for. Suite green. The dataset is not edited — its `pack:` field still names `/tmp/probe/longform.yaml`, because that is what produced it. ## Two things worth saying plainly **A red `main` blocks every open branch and skips the image publish** — #305 and #307 both measured that cost. I introduced the guard, so clearing it quickly is mine, and I would rather ship this than leave it for whoever notices next. **The guard behaved as intended rather than as a nuisance, and I would say so even if it had cost more.** A dataset arrived whose method was already unreachable. Without the check that becomes invisible the moment `/tmp` is cleaned, and the numbers in #236's evidence would join the twelve I spent this hour recovering. The gap was visible the same hour instead of next week. That is the argument for the check, made by the check. Whether a missing pack should redden `main` or only warn is a fair question, and it is not mine — if the answer is warn, the test should move to a non-gating report rather than lose the coverage. Refs #313

coilyco-ops added 1 commit

2026-08-13 12:32:51 +00:00

fix(evidence): preserve the longform pack, which reddened main on arrival

ci / test (pull_request) Successful in 37s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

ci / image-build (pull_request) Successful in 3s

Details

2cde1290bd

TestEveryCitedPackIsPreserved landed in c8e3336 and caught its first real case
within minutes: 0cb3c80 added probe-long-form-multi-turn.yaml citing
/tmp/probe/longform.yaml, which is scratch and not in the repository.

My guard is what turned main red, so unreddening it is mine. The pack still
existed on the machine that produced it, so the evidence is intact and this is
the one-file remedy the failure message asks for.

The dataset is not edited. Its pack field still names /tmp/probe/longform.yaml,
because that is what produced it.

Worth recording that the guard behaved as intended rather than as a nuisance. A
dataset arrived whose method was already unreachable, and the gap was visible
the same hour instead of after the machine was cleaned. That is the whole
argument for the check, made by the check.

Refs sirens-echo#313

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

coilyco-ops requested review from coilysiren

2026-08-13 12:32:51 +00:00

coilyco-ops merged commit 74ba903244 into main

2026-08-13 12:34:19 +00:00

coilyco-ops referenced this pull request from a commit

2026-08-13 12:34:20 +00:00

Merge pull request 'fix(evidence): preserve the longform pack, which reddened main on arrival' (#421) from qa/preserve-the-longform-pack into main

coilyco-ops referenced this pull request

2026-08-13 12:37:04 +00:00

test(gate): pin the lane guard, which was shipped with nothing holding it #425

coilyco-ops referenced this pull request

2026-08-13 12:38:20 +00:00