main has been red since 2026-08-07 and no release has been cut in 6 days #274

New issue

Closed

opened 2026-08-12 09:42:03 +00:00 by coilyco-ops · 0 comments

coilyco-ops commented

2026-08-12 09:42:03 +00:00

Member

Outcome

main is green and the release train is moving again.

State

TestLatestScoredResultsMatchCurrentPacks fails for every Core Roster codex pack: creator, design, director, engineer, exec, ops, qa.

result_test.go:58: result pack digest
  "sha256:8883035579629ad85134ea3064a25c68df78523e08119d28fe3c11e1410f0024"
  does not match
  "sha256:01c02b48e50c24f788713599709b6b3888c04beb50f6f94909152c053636a0f9"

Reproduces locally on current main with go test ./internal/evaluation/, so it is deterministic rather than a runner problem.

How long

Last green run: 202, commit 1e5c409c, 2026-08-07.
First red run: 203, commit 53be5372 - "feat(person): add shared role melds for cross-role doctrine (#231)".
Every run since has failed. That is 27 consecutive red runs and 60 commits.

Impact

release.yml runs tests and the release in one job, so a failed test means no release. The newest tag is v2.19.0, 2026-08-06 - cut the day before this broke. Every commit since is unreleased, including the roster rename e5c3b93 ("rename the strats role to exec, Executive Strategist"), which is a breaking change users have not received.

The failure is also silent in practice. The Telegram alert step reports telegram alert missing required secret and exits 2 under continue-on-error, so nothing has been paging about six days of red main.

Likely cause

The committed evaluation baselines under evaluations/latest record a pack digest, and ValidateResult compares it against a freshly Build-ed pack. Anything that changes pack content invalidates every recorded result at once, which matches all seven roles failing together rather than one.

53be5372 added shared role melds, and e5c3b93 later renamed a role. Both change what a pack contains. Marking this as inference: I confirmed the failure and its first commit, not that the baselines were skipped deliberately.

If that is right, the fix is to re-earn the baseline against the current packs:

ward exec evaluation-baseline
ward exec evaluation-scorecard

That verb is described as driving, independently reviewing, and recording every active case, so it is not a rubber stamp and needs a human to run it deliberately.

Do

Confirm the digest mismatch is the intended consequence of the pack changes, not a real regression in Build.
Re-earn the baseline and refresh the scorecard.
Fix the alert step so a red main actually pages. telegram alert missing required secret means the secrets are absent or unreadable in this repo. See coilysiren/inbox#354, which converts this repo's inlined alert body to the tracked script.
Consider whether a six-day silent red main should be catchable another way, given the alert path was the thing that failed.

Acceptance

go test ./... passes on main.
A release is cut, carrying the 60 unreleased commits.
A failing main produces an alert that arrives.

Origin

Found while landing the ## Checkout residency docs section (coilysiren/inbox#349). That change is docs-only and does not touch this. It was merged into an already-red main and fails the same way, which is how the pre-existing failure surfaced.

## Outcome `main` is green and the release train is moving again. ## State `TestLatestScoredResultsMatchCurrentPacks` fails for every Core Roster codex pack: creator, design, director, engineer, exec, ops, qa. ``` result_test.go:58: result pack digest "sha256:8883035579629ad85134ea3064a25c68df78523e08119d28fe3c11e1410f0024" does not match "sha256:01c02b48e50c24f788713599709b6b3888c04beb50f6f94909152c053636a0f9" ``` Reproduces locally on current `main` with `go test ./internal/evaluation/`, so it is deterministic rather than a runner problem. ## How long * Last green run: **202**, commit `1e5c409c`, 2026-08-07. * First red run: **203**, commit `53be5372` - "feat(person): add shared role melds for cross-role doctrine (#231)". * Every run since has failed. That is **27 consecutive red runs and 60 commits**. ## Impact `release.yml` runs tests and the release in one job, so a failed test means no release. The newest tag is **`v2.19.0`, 2026-08-06** - cut the day before this broke. Every commit since is unreleased, including the roster rename `e5c3b93` ("rename the strats role to exec, Executive Strategist"), which is a breaking change users have not received. The failure is also silent in practice. The Telegram alert step reports `telegram alert missing required secret` and exits 2 under `continue-on-error`, so nothing has been paging about six days of red main. ## Likely cause The committed evaluation baselines under `evaluations/latest` record a pack digest, and `ValidateResult` compares it against a freshly `Build`-ed pack. Anything that changes pack content invalidates every recorded result at once, which matches all seven roles failing together rather than one. `53be5372` added shared role melds, and `e5c3b93` later renamed a role. Both change what a pack contains. Marking this as inference: I confirmed the failure and its first commit, not that the baselines were skipped deliberately. If that is right, the fix is to re-earn the baseline against the current packs: ```sh ward exec evaluation-baseline ward exec evaluation-scorecard ``` That verb is described as driving, independently reviewing, and recording every active case, so it is not a rubber stamp and needs a human to run it deliberately. ## Do 1. Confirm the digest mismatch is the intended consequence of the pack changes, not a real regression in `Build`. 2. Re-earn the baseline and refresh the scorecard. 3. Fix the alert step so a red `main` actually pages. `telegram alert missing required secret` means the secrets are absent or unreadable in this repo. See coilysiren/inbox#354, which converts this repo's inlined alert body to the tracked script. 4. Consider whether a six-day silent red `main` should be catchable another way, given the alert path was the thing that failed. ## Acceptance * `go test ./...` passes on `main`. * A release is cut, carrying the 60 unreleased commits. * A failing `main` produces an alert that arrives. ## Origin Found while landing the `## Checkout residency` docs section (coilysiren/inbox#349). That change is docs-only and does not touch this. It was merged into an already-red `main` and fails the same way, which is how the pre-existing failure surfaced.

coilysiren closed this issue

2026-08-12 09:58:14 +00:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

coilyco-flight-deck/agent-compose#274

No description provided.

Rows
Columns