Run the coalescing rate case that has never been measured #1133

New issue

Open

opened 2026-08-23 01:19:57 +00:00 by coilyco-ops · 0 comments

coilyco-ops commented

2026-08-23 01:19:57 +00:00

Member

Split out of #988, whose structural item landed in #1122 and #1124 and whose remaining two need surfaces a code session does not hold.

The row that has never been run

agents/deep/packs/rate.yaml, case coalesced-ask-answers-every-comment:

observed: "NOT YET MEASURED. Added with the coverage line in sirens-echo#977 and never run,
           so this row is a target rather than a rate. Read it as unmeasured until someone
           replaces this line with a run."
runs: 10
max_failure_rate: 0.0
forbid_tool_call_markup: true

Three sums in one ask, comments: 3, one required pattern per answer, so a dropped comment is a missing number. This is the cheapest real evidence the coalescing lane has available and it needs no Discord, because it goes through the production prompt and Agent Proxy.

Until it runs, the coverage line in #983 is a plausible fix rather than a measured one, and the concern that motivated it, a smaller model answering the last comment and dropping the rest, is unresolved in either direction.

What it needs

just rate-deep, with AGENT_PROXY_URL, AGENT_PROXY_MODEL naming Deep's route, OTEL_EXPORTER_OTLP_ENDPOINT, and SIRENS_ECHO_MCP_ROSTER. Record host state in SIRENS_ECHO_SUBSTRATE first. All four were unset in the session that filed this, and the proxy is in-cluster, so this wants a host that can reach it.

Replace the observed line with the run rather than leaving the target.

The six live observations stay on #988

Those need a guild and are first-run behaviour that no unit test reaches.

Refs #988, #983, #977

Split out of #988, whose structural item landed in #1122 and #1124 and whose remaining two need surfaces a code session does not hold. ## The row that has never been run `agents/deep/packs/rate.yaml`, case `coalesced-ask-answers-every-comment`: ``` observed: "NOT YET MEASURED. Added with the coverage line in sirens-echo#977 and never run, so this row is a target rather than a rate. Read it as unmeasured until someone replaces this line with a run." runs: 10 max_failure_rate: 0.0 forbid_tool_call_markup: true ``` Three sums in one ask, `comments: 3`, one required pattern per answer, so a dropped comment is a missing number. **This is the cheapest real evidence the coalescing lane has available and it needs no Discord**, because it goes through the production prompt and Agent Proxy. Until it runs, the coverage line in #983 is a plausible fix rather than a measured one, and the concern that motivated it, a smaller model answering the last comment and dropping the rest, is unresolved in either direction. ## What it needs `just rate-deep`, with `AGENT_PROXY_URL`, `AGENT_PROXY_MODEL` naming Deep's route, `OTEL_EXPORTER_OTLP_ENDPOINT`, and `SIRENS_ECHO_MCP_ROSTER`. Record host state in `SIRENS_ECHO_SUBSTRATE` first. All four were unset in the session that filed this, and the proxy is in-cluster, so this wants a host that can reach it. Replace the `observed` line with the run rather than leaving the target. ## The six live observations stay on #988 Those need a guild and are first-run behaviour that no unit test reaches. Refs #988, #983, #977

coilyco-ops added this to the Headless engineer lane: the queue wait first milestone

2026-08-23 01:19:57 +00:00

coilyco-ops referenced this issue

2026-08-23 01:20:19 +00:00

Testing the coalescing lane owes: a seam for the Discord adapter, one measured rate, and six live observations #988

coilyco-ops removed this from the Headless engineer lane: the queue wait first milestone

2026-08-23 01:20:58 +00:00

coilyco-ops referenced this issue

2026-08-23 01:22:35 +00:00

Testing the coalescing lane owes: a seam for the Discord adapter, one measured rate, and six live observations #988

coilyco-ops referenced this issue

2026-08-23 01:42:07 +00:00