test(telemetry): the outage state, which is the one that was wrong #585

Merged

coilyco-ops merged 1 commit from qa/pin-the-outage-state into main

2026-08-13 16:33:20 +00:00

coilyco-ops commented

2026-08-13 16:29:23 +00:00

Member

The guard I shipped in #541 asserts a listing against a cache hit. #540 was neither — a failed connect spends a round trip and completes no listing, and it reported as a hit.

The fix added mcp.tools.reached and derived cached from it instead of from listed. That is the field carrying the correctness of the whole thing, and nothing asserted it.

Three states, one table

live, first open        cached=false  reached=1  listed=1
live, inside interval   cached=true   reached=0  listed=0
unreachable server      cached=false  reached=1  listed=0

A field is only worth anything if the cases it separates actually report differently, so all three go in one test rather than three green tests that could each pass while the field said the same thing.

One helper change

listingSpanAttributes no longer calls t.Fatalf when Open errors. An unreachable roster is one of the states under test rather than a broken fixture, and the span is written before the all-unavailable guard returns — which is precisely why the wrong value was observable in the first place.

Mutation checked four ways

Each fails naming the state that broke:

cached derived from listed again    the outage row fails
  -> "an outage was not reported as one …: map[mcp.tools.cached:true … reached:1]"

reached renamed                     both live rows fail
reached always incremented          the cache hit fails
a failed connect not counted        the outage row fails

The first is #540 exactly — revert the one-line derivation and this test names the regression and the map that proves it.

What this does not cover

#533. These attributes are correct and, on sirens-echo, they are correct on a parentless single-span trace that no turn can reach. A right answer in an unreachable place is still the open half.

Test-only. No production change.

Refs #540

The guard I shipped in #541 asserts a listing against a cache hit. **https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/540 was neither** — a failed connect spends a round trip and completes no listing, and it reported as a hit. The fix added `mcp.tools.reached` and derived `cached` from it instead of from `listed`. That is the field carrying the correctness of the whole thing, and **nothing asserted it**. ## Three states, one table ``` live, first open cached=false reached=1 listed=1 live, inside interval cached=true reached=0 listed=0 unreachable server cached=false reached=1 listed=0 ``` A field is only worth anything if the cases it separates actually report differently, so all three go in one test rather than three green tests that could each pass while the field said the same thing. ## One helper change `listingSpanAttributes` no longer calls `t.Fatalf` when `Open` errors. An unreachable roster is **one of the states under test** rather than a broken fixture, and the span is written before the all-unavailable guard returns — which is precisely why the wrong value was observable in the first place. ## Mutation checked four ways Each fails naming the state that broke: ``` cached derived from listed again the outage row fails -> "an outage was not reported as one …: map[mcp.tools.cached:true … reached:1]" reached renamed both live rows fail reached always incremented the cache hit fails a failed connect not counted the outage row fails ``` The first is https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/540 exactly — revert the one-line derivation and this test names the regression and the map that proves it. ## What this does not cover **https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/533.** These attributes are correct and, on `sirens-echo`, they are correct on a parentless single-span trace that no turn can reach. A right answer in an unreachable place is still the open half. Test-only. No production change. Refs https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/540

coilyco-ops added 1 commit

2026-08-13 16:29:23 +00:00

test(telemetry): the outage state, which is the one that was wrong

ci / test (pull_request) Successful in 30s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

ci / image-build (pull_request) Successful in 21s

Details

40f8b961c2

The guard I shipped in #541 asserts a listing against a cache hit. sirens-echo#540
was neither: a failed connect spends a round trip and completes no listing, and
it reported as a hit. The fix added mcp.tools.reached and derived cached from
it, and nothing asserts that field.

Three states in one table, since a field is only worth anything if the cases it
separates report differently:

  live, first open      cached=false  reached=1  listed=1
  live, inside interval cached=true   reached=0  listed=0
  unreachable server    cached=false  reached=1  listed=0

The helper no longer fails the test when Open errors. An unreachable roster is
one of the states under test rather than a broken fixture, and the span is
written before the all-unavailable guard returns.

Mutation checked four ways, each caught naming the state that broke:

  cached derived from listed again   the outage row fails, which is 540 exactly
  reached renamed                    both live rows fail
  reached always incremented         the cache hit fails
  a failed connect not counted       the outage row fails

Refs sirens-echo#540

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

coilyco-ops requested review from coilysiren

2026-08-13 16:29:23 +00:00

coilyco-ops merged commit 76650c311e into main

2026-08-13 16:33:20 +00:00

coilyco-ops referenced this pull request from a commit

2026-08-13 16:33:21 +00:00

Merge pull request 'test(telemetry): the outage state, which is the one that was wrong' (#585) from qa/pin-the-outage-state into main