test(access): prove the exit codes deploy's CI keys on #654

Merged

coilyco-ops merged 1 commit from test/the-access-check-exit-codes-claude into main

2026-08-13 17:58:49 +00:00

coilyco-ops commented

2026-08-13 17:57:45 +00:00

Member

closes #653

The gate from #628 printed failures and exited, and nothing asserted which code. Deploy's CI reads the exit status, not the text — so a run that reports a bad policy on stderr and exits 0 is worse than no gate at all, which is the exact failure #628 exists to prevent.

main now returns from run instead of calling os.Exit inside the loop. That is the smallest change that makes the codes assertable.

Four cases

a valid policy exits 0 and confirms on stdout
an invalid one exits 1, reason on stderr, nothing on stdout
no arguments exits 2
one bad file among good ones fails the whole run — the case that matters for a CI step handed a directory, and the one where a per-file loop most easily returns success

Both mutations run

return 2 -> 0   "no arguments exited 0, want 2"
return 1 -> 0   "a bad policy exited 0, want 1"
                "a run with one bad file exited 0, want 1"

None pass for a reason unrelated to the code they assert.

Why the issue exists alongside this

While mutation-testing I checked results with grep -c "\-\-\- FAIL" and got 0, and read it as "mutation not caught". The package was not compiling — a refactor had been reverted while the test file still referenced the new function. A compile error is not a --- FAIL line, so the grep could not see the worst outcome and reported it as the best one.

I nearly went looking for a weakness in a test that was fine. That is #592's shape again — zero failures and zero-because-nothing-ran are the same output — and it is recorded rather than remembered as a private slip.

ward exec gate green.

closes #653 The gate from #628 printed failures and exited, and **nothing asserted which code**. Deploy's CI reads the exit status, not the text — so a run that reports a bad policy on stderr and exits 0 is worse than no gate at all, which is the exact failure #628 exists to prevent. `main` now returns from `run` instead of calling `os.Exit` inside the loop. That is the smallest change that makes the codes assertable. ## Four cases - a valid policy exits **0** and confirms on stdout - an invalid one exits **1**, reason on stderr, nothing on stdout - no arguments exits **2** - **one bad file among good ones fails the whole run** — the case that matters for a CI step handed a directory, and the one where a per-file loop most easily returns success ## Both mutations run ``` return 2 -> 0 "no arguments exited 0, want 2" return 1 -> 0 "a bad policy exited 0, want 1" "a run with one bad file exited 0, want 1" ``` None pass for a reason unrelated to the code they assert. ## Why the issue exists alongside this While mutation-testing I checked results with `grep -c "\-\-\- FAIL"` and got **0**, and read it as "mutation not caught". The package was not compiling — a refactor had been reverted while the test file still referenced the new function. **A compile error is not a `--- FAIL` line**, so the grep could not see the worst outcome and reported it as the best one. I nearly went looking for a weakness in a test that was fine. That is #592's shape again — zero failures and zero-because-nothing-ran are the same output — and it is recorded rather than remembered as a private slip. `ward exec gate` green.

coilyco-ops added 1 commit

2026-08-13 17:57:46 +00:00

test(access): prove the exit codes deploy's CI keys on

ci / image-build (pull_request) Successful in 21s

Details

ci / test (pull_request) Successful in 35s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

8ed482841f

The command printed failures and exited, and nothing asserted which code. A
gate that reports a bad policy on stderr and exits 0 is worse than no gate,
because CI reads the code and not the text.

main now returns from run rather than calling os.Exit inside the loop, which
is the smallest change that makes the codes assertable at all.

Four cases: valid exits 0 and confirms on stdout, invalid exits 1 with the
reason on stderr and nothing on stdout, no arguments exits 2, and one bad file
among good ones fails the whole run. The last is the one that matters for a CI
step handed a directory.

Both mutations run. Returning 0 from the failure path fails two tests and
returning 0 from the usage path fails the third, so none of them pass for a
reason unrelated to the code they assert.

Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

coilyco-ops requested review from coilysiren

2026-08-13 17:57:46 +00:00

coilyco-ops merged commit 0ccd7a3e90 into main

2026-08-13 17:58:49 +00:00

coilyco-ops referenced this pull request from a commit

2026-08-13 17:58:49 +00:00

Merge pull request 'test(access): prove the exit codes deploy's CI keys on' (#654) from test/the-access-check-exit-codes-claude into main