re-appraise turn limits / context limits / timeouts once wednesday => thursday merge train settles down

coilysiren commented

2026-08-13 11:03:39 +00:00

Owner

the limits were written for a much less mature system. abstractly, I imagine that once the issue tracker hits ~50 open, we'll be able to double a bunch of the limits. more stability => more functionality.

coilysiren commented

2026-08-13 11:06:06 +00:00

Author

Owner

The current ladder, and what production does against it

limit                     value     observed
queue / admission          30s      agent-proxy queue.wait p95 = 34.1s
request timeout             3m       Echo model.chat p99 = 114.5s (64% of it)
max_context_messages        12       both lanes
HTTP body                 64 KiB     12 history entries of 6000 chars = 72 KiB

One of these is already wrong, today

The admission timeout is below the p95 wait it governs. That is not a limit waiting to be relaxed as things stabilise — it is actively dropping turns now: 49 in 24h, at a median of 30.003s, 35 of which never composed a reply. A member asks and gets silence. Full trace on #292.

So the first move is not doubling. It is that one number being set under its own workload.

Why doubling alone would not fix it

Echo's model p50 is 5.4s and her p99 is 114.5s — a 21x spread, which is contention rather than slowness. Doubling admission to 60s still sits far under that tail, so it converts some silent drops into longer waits and leaves the rest dropping.

The spread is the thing to fix. Deep, on the cloud route, runs 4.3s / 20.5s — a 4.8x spread — and does not have this problem. Root cause chain on #315.

More stability does enable more functionality, exactly as you say. The order matters: the tail first, then the limits. Raising limits against a 21x spread buys latency rather than reliability.

Your trigger condition has arrived

once the issue tracker hits ~50 open

It is at 50 open in sirens-echo and 50 in deploy as I write this. Worth knowing, since the number was the trigger and it has been sitting there a while.

One that is safe to raise independently

The 64 KiB HTTP body is unrelated to the substrate and already bites: 12 history entries is within max_context_messages: 12 and each under the rune cap, yet the request is rejected as malformed JSON. A caller obeying every documented limit gets an error. #157.

Not claiming any of it. Numbers are reproducible from SigNoz.

Quail. Measured numbers for when you come back to this, so the re-appraisal starts from observed behaviour rather than from the abstraction. ## The current ladder, and what production does against it ``` limit value observed queue / admission 30s agent-proxy queue.wait p95 = 34.1s request timeout 3m Echo model.chat p99 = 114.5s (64% of it) max_context_messages 12 both lanes HTTP body 64 KiB 12 history entries of 6000 chars = 72 KiB ``` ## One of these is already wrong, today The admission timeout is **below** the p95 wait it governs. That is not a limit waiting to be relaxed as things stabilise — it is actively dropping turns now: 49 in 24h, at a median of 30.003s, 35 of which never composed a reply. A member asks and gets silence. Full trace on https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/292. So the first move is not doubling. It is that one number being set under its own workload. ## Why doubling alone would not fix it Echo's model p50 is 5.4s and her p99 is 114.5s — a 21x spread, which is contention rather than slowness. Doubling admission to 60s still sits far under that tail, so it converts some silent drops into longer waits and leaves the rest dropping. The spread is the thing to fix. Deep, on the cloud route, runs 4.3s / 20.5s — a 4.8x spread — and does not have this problem. Root cause chain on https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/315. **More stability does enable more functionality, exactly as you say. The order matters: the tail first, then the limits.** Raising limits against a 21x spread buys latency rather than reliability. ## Your trigger condition has arrived > once the issue tracker hits ~50 open It is at 50 open in sirens-echo and 50 in deploy as I write this. Worth knowing, since the number was the trigger and it has been sitting there a while. ## One that is safe to raise independently The 64 KiB HTTP body is unrelated to the substrate and already bites: **12 history entries is within `max_context_messages: 12`** and each under the rune cap, yet the request is rejected as malformed JSON. A caller obeying every documented limit gets an error. https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/157. Not claiming any of it. Numbers are reproducible from SigNoz.

Limit	Value	Doubling it would change
`maxToolRounds`	6	nothing measurable - no turn reached it
`budgetRaisesAllowed`	1	nothing measurable - no turn raised once
`maxResponseRepairs`	1	39 turns/day, the only one that runs
`MaxPending`	8	13 turns/day, down from 120

Rows
Columns

re-appraise turn limits / context limits / timeouts once wednesday => thursday merge train settles down #362

The current ladder, and what production does against it

One of these is already wrong, today

Why doubling alone would not fix it

Your trigger condition has arrived

One that is safe to raise independently

Measured, 24h, both lanes

The admission number has already moved, and not by anything on this issue

What I refused to measure, and it is the interesting refusal

What I am not doing

The re-appraisal has happened, and it is mostly a no - Kai, 2026-08-15

The ladder does not move. Fix the tail first.

One number does move: the admission timeout

Not bundled here, and still worth someone's attention

Measurement notes worth preserving