coilyco-gaming/sirens-echo

Watch

Carry-forward: performance and reliability findings measured during the August 19 demo, preserved after the readiness P0s closed #1094

New issue

Open

opened 2026-08-22 20:42:01 +00:00 by coilyco-ops · 1 comment

coilyco-ops commented

2026-08-22 20:42:01 +00:00

Member

Source issues, all closed as delivered: #981 (epic), #1037, #1010, #1002, #993.

Measurements worth keeping

Prefix cost - a 116 KB prefix was paid roughly six times per reply, inside a 32.9s turn. Measured in #1002. Nothing about this is demo-specific, and it is the single largest known contributor to turn latency.
Skill root disclosure - Dowel's skill root opts out of progressive disclosure entirely, inlining all six references at 24 KB on every turn. Measured in #993. This compounds directly into the prefix cost above.
Execution slot exhaustion - Dowel refused 3 of 8 summons under the stream's own arrival shape, each refusal arriving only after a 50 second wait for an execution slot. Measured in #1010. The related slot-pool work is #995, which remains open at P1 and headless.
Member-facing error rate - #1037 asked to measure and cut it while keeping trace IDs. No baseline number was recorded before the issue closed, so this one survives as an open question rather than a measurement.

Why this is not simply reopened

The readiness framing was correct for a dated stream and is wrong now. These findings should be re-ranked on their own merits against whatever forcing event comes next, not inherited at P0 from an event that has passed.

Open question for whoever picks this up

#1037 never produced a baseline error rate. Without it there is no way to tell whether the demo lane's reliability has since improved, regressed, or held. Establishing that baseline is the cheapest next step and it gates any claim about the other three.

The August 19 Temporal Vibe Check readiness issues were closed as delivered on 2026-08-22 once the stream shipped. Several of them carried measurements that are properties of the harness rather than of the demo, so they are recorded here instead of being lost with the closes. Source issues, all closed as delivered: #981 (epic), #1037, #1010, #1002, #993. ## Measurements worth keeping * Prefix cost - a 116 KB prefix was paid roughly six times per reply, inside a 32.9s turn. Measured in #1002. Nothing about this is demo-specific, and it is the single largest known contributor to turn latency. * Skill root disclosure - Dowel's skill root opts out of progressive disclosure entirely, inlining all six references at 24 KB on every turn. Measured in #993. This compounds directly into the prefix cost above. * Execution slot exhaustion - Dowel refused 3 of 8 summons under the stream's own arrival shape, each refusal arriving only after a 50 second wait for an execution slot. Measured in #1010. The related slot-pool work is #995, which remains open at P1 and headless. * Member-facing error rate - #1037 asked to measure and cut it while keeping trace IDs. No baseline number was recorded before the issue closed, so this one survives as an open question rather than a measurement. ## Why this is not simply reopened The readiness framing was correct for a dated stream and is wrong now. These findings should be re-ranked on their own merits against whatever forcing event comes next, not inherited at P0 from an event that has passed. ## Open question for whoever picks this up #1037 never produced a baseline error rate. Without it there is no way to tell whether the demo lane's reliability has since improved, regressed, or held. Establishing that baseline is the cheapest next step and it gates any claim about the other three.

coilyco-ops referenced this issue

2026-08-22 20:42:08 +00:00

Epic: August 19 Temporal Vibe Check demo readiness, the work that is on camera #981

coilyco-ops referenced this issue

2026-08-22 20:42:11 +00:00

Measure and cut the demo lane's member-facing error rate, keeping trace IDs #1037

coilyco-ops referenced this issue

2026-08-22 20:42:13 +00:00

Load test: Dowel refuses 3 of 8 summons at the stream's own arrival shape, each after a 50s wait for an execution slot #1010

coilyco-ops referenced this issue

2026-08-22 20:42:15 +00:00

Measure where the 32.9s turn actually goes: a 116 KB prefix paid roughly six times per reply #1002

coilyco-ops referenced this issue

2026-08-22 20:42:17 +00:00

Dowel's skill root opts out of progressive disclosure entirely: all six references are inline, 24KB every turn #993

coilyco-ops referenced this issue

2026-08-22 20:55:26 +00:00

Lane: headless engineer burn-down in sirens-echo, 19 issues at P1 and P2, led by the slot pool the load test already indicted #1095

coilyco-ops commented

2026-08-23 01:39:59 +00:00

Author

Member

Establishing the baseline this issue records as missing. #1095 notes that no member-facing error rate was ever captured, so there is no way to tell whether this lane's reliability improved, regressed, or held, and calls it cheap and gating on any claim the lane makes about its own results.

Here it is. SigNoz, read-only, sirens_echo.turns by outcome, seven days to 2026-08-23, temporality: delta explicit.

lane	error share of turns
`sirens-dowel`	5.5%
`sirens-deep`	3.9%
`sirens-deep-owl-glass`	2.1%
`sirens-echo`	57.1%
all lanes	5.7%

How to read it

sirens-deep at 3.9% is the number worth carrying forward. It is the lane still taking traffic and the one with a meaningful sample, 78 turns.

sirens-echo's 57.1% is not a reliability signal. It saw 11 turns in seven days, of which its own p50 turn duration is 300.8s. A majority-error rate over that sample says the lane is barely used and its few turns are slow, not that the harness fails half the time.

sirens-dowel's 5.5% is historical. It ran 164 turns across 2026-08-18 and 08-19 and has produced nothing since, consistent with coilyco-bridge/deploy#718.

The caveat that matters for later comparisons

This is a turn-outcome rate, not a member-experience rate. It counts turns the harness recorded as error. It does not count:

a summon shed at admission before a turn exists, which is 6.1% of admissions separately
a turn a restart took, which had no record at all until #989 landed today
a message that never summoned, which had no record until #992 landed today

So a later measurement taken against this one is only comparable on the same definition, and two of those three gaps were closed today, which means the next window may show a higher number from better recording rather than from worse behaviour. Worth knowing before anyone reads a rise as a regression.

Refs #1095, coilyco-bridge/deploy#718, #989, #992

**Establishing the baseline this issue records as missing.** #1095 notes that no member-facing error rate was ever captured, so there is no way to tell whether this lane's reliability improved, regressed, or held, and calls it cheap and gating on any claim the lane makes about its own results. Here it is. SigNoz, read-only, `sirens_echo.turns` by outcome, seven days to 2026-08-23, `temporality: delta` explicit. | lane | error share of turns | | --- | --- | | `sirens-dowel` | 5.5% | | `sirens-deep` | **3.9%** | | `sirens-deep-owl-glass` | 2.1% | | `sirens-echo` | 57.1% | | **all lanes** | **5.7%** | ## How to read it **`sirens-deep` at 3.9% is the number worth carrying forward.** It is the lane still taking traffic and the one with a meaningful sample, 78 turns. **`sirens-echo`'s 57.1% is not a reliability signal.** It saw 11 turns in seven days, of which its own p50 turn duration is 300.8s. A majority-error rate over that sample says the lane is barely used and its few turns are slow, not that the harness fails half the time. **`sirens-dowel`'s 5.5% is historical.** It ran 164 turns across 2026-08-18 and 08-19 and has produced nothing since, consistent with coilyco-bridge/deploy#718. ## The caveat that matters for later comparisons This is a **turn-outcome** rate, not a member-experience rate. It counts turns the harness recorded as `error`. It does **not** count: * a summon shed at admission before a turn exists, which is 6.1% of admissions separately * a turn a restart took, which had no record at all until #989 landed today * a message that never summoned, which had no record until #992 landed today So a later measurement taken against this one is only comparable on the same definition, and **two of those three gaps were closed today**, which means the next window may show a higher number from better recording rather than from worse behaviour. Worth knowing before anyone reads a rise as a regression. Refs #1095, coilyco-bridge/deploy#718, #989, #992

coilyco-ops referenced this issue

2026-08-23 01:40:47 +00:00

Lane: headless engineer burn-down in sirens-echo, 19 issues at P1 and P2, led by the slot pool the load test already indicted #1095

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

coilyco-gaming/sirens-echo#1094

No description provided.

Rows
Columns