test(rate): pin the half of the cut-run fix that stops the work #338

Merged

coilysiren merged 1 commit from qa/cut-run-stops-calling into main

2026-08-13 10:22:39 +00:00

coilyco-ops commented

2026-08-13 10:15:46 +00:00

Member

Verifying #327 by mutation rather than by reading it. The fix is correct; one of its two halves is untested.

e2249dc does two things: the dataset survives a cut run, and the loop stops calling the model once the context is cancelled. TestRateEmitsWhatItMeasuredWhenTheRunIsCut covers the first.

It passes with the loop break removed

instantCompletions never blocks, so a run that ignores the cancellation finishes every case instantly, still ends with a non-nil ctx.Err(), and still writes a dataset with provenance. Every assertion in the existing test holds.

Measured with a counting client:

main                  0 model calls after cancel
loop break removed    2 model calls, one per case in the pack

Why this half is the one that matters operationally

The commit exists because a wedged route cost ten minutes. Without the break, a cancelled run pays that wait once more per remaining case — the interrupt stops being an escape and becomes a delayed exit. The dataset would still be written, so the visible symptom the existing test guards would look fine while the operator waits.

The test

A countingCompletions client and one assertion: a cancelled run makes zero model calls. Verified by removing the break, which now fails with a cancelled run made 2 model calls, want 0.

Tests only. No production code changes.

Verifying https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/pulls/327 by mutation rather than by reading it. **The fix is correct; one of its two halves is untested.** e2249dc does two things: the dataset survives a cut run, and the loop stops calling the model once the context is cancelled. `TestRateEmitsWhatItMeasuredWhenTheRunIsCut` covers the first. ## It passes with the loop break removed `instantCompletions` never blocks, so a run that ignores the cancellation finishes every case instantly, still ends with a non-nil `ctx.Err()`, and still writes a dataset with provenance. Every assertion in the existing test holds. Measured with a counting client: ``` main 0 model calls after cancel loop break removed 2 model calls, one per case in the pack ``` ## Why this half is the one that matters operationally The commit exists because a wedged route cost ten minutes. Without the break, a cancelled run pays that wait **once more per remaining case** — the interrupt stops being an escape and becomes a delayed exit. The dataset would still be written, so the visible symptom the existing test guards would look fine while the operator waits. ## The test A `countingCompletions` client and one assertion: a cancelled run makes zero model calls. Verified by removing the break, which now fails with `a cancelled run made 2 model calls, want 0`. Tests only. No production code changes.

coilyco-ops added 1 commit

2026-08-13 10:15:46 +00:00

test(rate): pin the half of the cut-run fix that stops the work

ci / test (pull_request) Successful in 30s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

ci / image-build (pull_request) Successful in 17s

Details

bafac63194

e2249dc has two halves. The dataset survives a cut run, and the loop stops
calling the model once the context is cancelled. Only the first is tested.

TestRateEmitsWhatItMeasuredWhenTheRunIsCut passes with the loop break
removed, because instantCompletions never blocks: a run that ignores the
cancellation finishes every case instantly, still ends with a non-nil
ctx.Err, and still writes a dataset. Every assertion holds.

Measured with a counting client:

  main                    0 model calls after cancel
  loop break removed      2 model calls, one per case in the pack

The untested half is the one that matters operationally. The commit was
written because a wedged route cost ten minutes; without the break, a
cancelled run pays that wait once more per remaining case.

Verified by removing the break, which now fails with the count.

Tests only. No production code changes.

Refs #327

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

coilyco-ops requested review from coilysiren

2026-08-13 10:15:46 +00:00