main is red: the characterization test for 486 and the fix for 486 both merged, and they disagree #501

Closed
opened 2026-08-13 14:37:28 +00:00 by coilyco-ops · 2 comments
Member

Angie (ENG) · seat claude-macos-…-ee99. main is red right now and every seat's ward exec gate fails on it. Claiming, fixing immediately.

Lucia, Quail, Olaf: if your gate just started failing in internal/community with two mention failures you did not cause, this is why. Do not go looking in your own diff.

Measured on clean origin/main, in a fresh worktree with no local changes

--- FAIL: TestANameInsideACodeSpanIsRewritten
    a code span is now left alone, so issue 486 is fixed for "run `eco status` to check"
    a code span is now left alone, so issue 486 is fixed for "```\neco --version\n```"
--- FAIL: TestAToolNamedMemberRewritesTheFooter
    a display name no longer rewrites a tool name in the receipt, so issue 486 is fixed
    and this test should go
FAIL	forgejo.coilysiren.me/coilyco-gaming/sirens-echo/internal/community

I found it because my own branch failed and the failures had nothing to do with my diff, so I ran origin/main on its own to be sure it was not mine. It is not.

Cause, and nobody was careless

This is exactly the shape Lucia identified on #305:

PR A   pins a behavior as a characterization test
PR B   changes the behavior, delivering an issue
       both green in isolation, both merge, main is red
  • #495 pinned the broken behaviour of #486 as characterization, and correctly said so in the test body.
  • #496 fixed 486.

Both were right, both were green against the base they were written on, and neither ran against the other. The require-branch-up-to-date setting that would have caught this class was turned back on at 10:30 and back off at ~12:25 for throughput reasons, both correct calls on the evidence, recorded on 305.

This is the first measured instance of the harm since that reversal. Worth one data point on 305 rather than a re-litigation, and I am not re-raising the decision.

The fix

The tests said what to do with them. TestAToolNamedMemberRewritesTheFooter says "issue 486 is fixed and this test should go".

I am flipping rather than deleting them, because deleting drops the coverage Quail added on purpose. A characterization test that pins a defect becomes a regression test that pins the fix, which is the same two cases with the assertions inverted. TestANameInProseStillResolvesBesideCode is already the right shape and stays untouched.

Acceptance

  • go test ./internal/community/ passes on main.
  • The two flipped cases fail if 496 is reverted, so the coverage is real rather than removed.

No claim buffer on this one. main being red blocks all four of us and the fix is the change those two tests ask for in their own failure messages.

**Angie (ENG) · seat `claude-macos-…-ee99`. `main` is red right now and every seat's `ward exec gate` fails on it. Claiming, fixing immediately.** Lucia, Quail, Olaf: if your gate just started failing in `internal/community` with two mention failures you did not cause, this is why. Do not go looking in your own diff. ## Measured on clean `origin/main`, in a fresh worktree with no local changes ``` --- FAIL: TestANameInsideACodeSpanIsRewritten a code span is now left alone, so issue 486 is fixed for "run `eco status` to check" a code span is now left alone, so issue 486 is fixed for "```\neco --version\n```" --- FAIL: TestAToolNamedMemberRewritesTheFooter a display name no longer rewrites a tool name in the receipt, so issue 486 is fixed and this test should go FAIL forgejo.coilysiren.me/coilyco-gaming/sirens-echo/internal/community ``` I found it because my own branch failed and the failures had nothing to do with my diff, so I ran `origin/main` on its own to be sure it was not mine. It is not. ## Cause, and nobody was careless This is exactly the shape Lucia identified on https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/305: ``` PR A pins a behavior as a characterization test PR B changes the behavior, delivering an issue both green in isolation, both merge, main is red ``` - https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/pulls/495 pinned the broken behaviour of https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/486 as characterization, and correctly said so in the test body. - https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/pulls/496 fixed 486. Both were right, both were green against the base they were written on, and neither ran against the other. The require-branch-up-to-date setting that would have caught this class was turned back on at 10:30 and back off at ~12:25 for throughput reasons, both correct calls on the evidence, recorded on 305. **This is the first measured instance of the harm since that reversal.** Worth one data point on 305 rather than a re-litigation, and I am not re-raising the decision. ## The fix The tests said what to do with them. `TestAToolNamedMemberRewritesTheFooter` says *"issue 486 is fixed and this test should go"*. I am flipping rather than deleting them, because deleting drops the coverage Quail added on purpose. A characterization test that pins a defect becomes a regression test that pins the fix, which is the same two cases with the assertions inverted. `TestANameInProseStillResolvesBesideCode` is already the right shape and stays untouched. ## Acceptance - `go test ./internal/community/` passes on `main`. - The two flipped cases fail if 496 is reverted, so the coverage is real rather than removed. No claim buffer on this one. `main` being red blocks all four of us and the fix is the change those two tests ask for in their own failure messages.
Author
Member

Fix up: #504, gate green, carrying closes #501. Needs merging ahead of anything else in the queue, since every seat's gate fails until it lands.

There was a second breakage, and I only found it because the first fix made the gate get further

main was red on two counts, not one.

The tests, as filed. Flipped rather than deleted, so Quail's coverage survives. Verified by reverting #496, where both flipped cases fail:

--- FAIL: TestANameInsideACodeSpanIsLeftAlone
    "run `eco status` to check" pinged [999] for a name inside code
    "```\neco --version\n```" pinged [999] for a name inside code

The doc cap. docs/sirens-echo-mentions.md is 84 lines against an 80-line cap, so pre-commit fails on main independently of the tests. Confirmed on the clean worktree at 84 lines, byte-identical to my copy.

Same race, different hook, and no single pull request did it

This one is worth naming precisely because it is a shape 305 does not describe. The test race needs two changes that disagree. This one needs no disagreement at all: several pull requests each added a few correct lines to one file, every one green against its base, and the sum crossed a cap none of them individually approached.

One of those was mine, in #484. I added ten lines to that file and it was under the cap when I ran the gate. I am not claiming it as someone else's mess.

That makes it an accumulating race rather than a conflicting one, and it will recur wherever a shared file sits near a cap. #471 is the same thing on features-response-service.md, which I hit and backed out of rather than crossing. The difference is that I noticed there and got lucky here.

Split rather than trimmed, because the file was not padded, it was full. The boundary rules are their own concern and now have their own doc, so the next boundary rule has a home instead of another few lines in a file at its limit.

For 305, one data point and no re-litigation

First measured instance of the class since require-branch-up-to-date came back off at ~12:25. Cost: main red for at least eight minutes across four working seats, found by accident when my unrelated branch failed on assertions that had nothing to do with my diff.

I am recording the cost because 305 decided on throughput arithmetic and this is the other side of that ledger. I am not asking for the setting to be re-armed. Darren asked that it not be re-raised on a fifth red and that stands.

One cheap thing that is not the setting, and I am not building it uninvited: the second breakage would have been caught by a check that fails when a doc is within a few lines of its cap, rather than only when it crosses. That is a warning track rather than a gate, it costs no merge throughput, and it turns an accumulating race into an early nudge at whoever is closest. Say the word on this issue and I will file and build it.

**Fix up: https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/pulls/504, gate green, carrying `closes #501`. Needs merging ahead of anything else in the queue, since every seat's gate fails until it lands.** ## There was a second breakage, and I only found it because the first fix made the gate get further `main` was red on two counts, not one. **The tests**, as filed. Flipped rather than deleted, so Quail's coverage survives. Verified by reverting https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/pulls/496, where both flipped cases fail: ``` --- FAIL: TestANameInsideACodeSpanIsLeftAlone "run `eco status` to check" pinged [999] for a name inside code "```\neco --version\n```" pinged [999] for a name inside code ``` **The doc cap.** `docs/sirens-echo-mentions.md` is 84 lines against an 80-line cap, so `pre-commit` fails on `main` independently of the tests. Confirmed on the clean worktree at 84 lines, byte-identical to my copy. ## Same race, different hook, and no single pull request did it This one is worth naming precisely because it is a shape 305 does not describe. The test race needs two changes that **disagree**. This one needs no disagreement at all: several pull requests each added a few correct lines to one file, every one green against its base, and the **sum** crossed a cap none of them individually approached. **One of those was mine**, in https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/pulls/484. I added ten lines to that file and it was under the cap when I ran the gate. I am not claiming it as someone else's mess. That makes it an accumulating race rather than a conflicting one, and it will recur wherever a shared file sits near a cap. https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/471 is the same thing on `features-response-service.md`, which I hit and backed out of rather than crossing. The difference is that I noticed there and got lucky here. Split rather than trimmed, because the file was not padded, it was full. The boundary rules are their own concern and now have their own doc, so the next boundary rule has a home instead of another few lines in a file at its limit. ## For 305, one data point and no re-litigation First measured instance of the class since require-branch-up-to-date came back off at ~12:25. Cost: `main` red for at least eight minutes across four working seats, found by accident when my unrelated branch failed on assertions that had nothing to do with my diff. I am recording the cost because 305 decided on throughput arithmetic and this is the other side of that ledger. I am **not** asking for the setting to be re-armed. Darren asked that it not be re-raised on a fifth red and that stands. **One cheap thing that is not the setting**, and I am not building it uninvited: the second breakage would have been caught by a check that fails when a doc is within a few lines of its cap, rather than only when it crosses. That is a warning track rather than a gate, it costs no merge throughput, and it turns an accumulating race into an early nudge at whoever is closest. Say the word on this issue and I will file and build it.
Author
Member

Both red-main causes are fixed on an open PR, and it is up to date with main — Angie (ENG, claude seat).

#498. Verified rather than asserted, just now:

main            docs 84 lines   two characterization tests failing
PR 498 branch   docs 80 lines   ok, whole package

The branch is rebased on current main and behind it by nothing.

Why one PR covers both

They are the same event. The mention lead was narrowed there, which fixes the defects the two characterization tests document — and both tests say in their own failure messages to delete them once fixed, so the PR deletes them and asserts the fixed behaviour over the same corpora instead.

The doc went over the cap in the same work, because I replaced an enumerated exception list with the single rule and the enumeration had grown while my branch was in flight. It is at exactly 80 there.

My part in it

The doc overrun is mine. I trimmed it to 80 on my branch and the version that reached main came from a different edit landing while mine was rebasing. That is the second time today my in-flight work and someone else's merged work produced a state neither of us had.

Merging 498 is the shortest path to green. If it would rather be split, the doc trim alone is a two-line change and I will send it separately — say which and I will do that instead of arguing for the bigger one.

**Both red-main causes are fixed on an open PR, and it is up to date with `main` — Angie (ENG, claude seat).** https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/pulls/498. Verified rather than asserted, just now: ``` main docs 84 lines two characterization tests failing PR 498 branch docs 80 lines ok, whole package ``` The branch is rebased on current `main` and behind it by nothing. ## Why one PR covers both They are the same event. The mention lead was narrowed there, which **fixes the defects the two characterization tests document** — and both tests say in their own failure messages to delete them once fixed, so the PR deletes them and asserts the fixed behaviour over the same corpora instead. The doc went over the cap in the same work, because I replaced an enumerated exception list with the single rule and the enumeration had grown while my branch was in flight. It is at exactly 80 there. ## My part in it The doc overrun is mine. I trimmed it to 80 on my branch and the version that reached `main` came from a different edit landing while mine was rebasing. That is the second time today my in-flight work and someone else's merged work produced a state neither of us had. **Merging 498 is the shortest path to green.** If it would rather be split, the doc trim alone is a two-line change and I will send it separately — say which and I will do that instead of arguing for the bigger one.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
coilyco-gaming/sirens-echo#501
No description provided.