No client-side rate limiting, which is a stated shared requirement for five servers about to be built on this runtime #57

New issue

Closed

opened 2026-08-13 16:20:58 +00:00 by coilyco-ops · 1 comment

coilyco-ops commented

2026-08-13 16:20:58 +00:00

Member

What this is

coilyco-bridge/deploy#465 commits to building five keyless media MCP servers on this runtime, and lists as a shared requirement applying to all five:

Client-side rate limiting, in the server. Limits are per IP and the pod has one, so every Echo user shares a bucket. MusicBrainz at ~1 req/sec especially. This is the failure that passes in testing and 503s under real channel traffic.

So the requirement cannot be met by any guardfile, and the first of the five — Open Library, coilyco-bridge/deploy#468 — shipped without it.

Why it matters more than it sounds

The consuming pod has one IP and many users behind it. A public-good API sees one caller whose request rate is the sum of a Discord community's curiosity, with no bucket in between. Open Library publishes no limit, which the deploy issue correctly reads as "not a licence to hammer a nonprofit's service."

The sharpest case is MusicBrainz (coilyco-bridge/deploy#470) at ~1 req/sec published. Two concurrent turns exceed it. There is currently no way to write a guardfile that does not.

What would be enough

A per-wrap bucket would cover every case in the batch:

wrap ward mcp musicbrainz {
    base-url "musicbrainz.org/ws/2"
    rate-limit 1/1s        // requests to the upstream, process-wide
}

Serialising at the outbound client rather than rejecting the caller is probably the right behaviour — a queued tool call is slower, a 503 is a failed turn. Per-grant granularity is not needed for this batch.

What I verified

Read against mcp-beaver main and cli-guard v0.131.0 on 2026-08-13. I did not attempt an implementation.

coilyco-bridge/deploy#465 — the epic carrying the requirement
coilyco-bridge/deploy#468 — Open Library, shipped without it
coilyco-bridge/deploy#470 — MusicBrainz, where this binds hardest

## What this is `coilyco-bridge/deploy#465` commits to building five keyless media MCP servers on this runtime, and lists as a shared requirement applying to all five: > **Client-side rate limiting, in the server.** Limits are **per IP** and the pod has one, so every Echo user shares a bucket. MusicBrainz at ~1 req/sec especially. This is the failure that passes in testing and 503s under real channel traffic. There is no such construct. Searching this repository for `rate.limit`, `ratelimit`, `throttle` returns nothing, and neither the wrap grammar (`base-url | auth | restrict | can | proxy`) nor the grant grammar (`path | query | body | set | fail-when`) has a place to express one. So the requirement cannot be met by any guardfile, and the first of the five — Open Library, `coilyco-bridge/deploy#468` — shipped without it. ## Why it matters more than it sounds The consuming pod has one IP and many users behind it. A public-good API sees one caller whose request rate is the *sum* of a Discord community's curiosity, with no bucket in between. Open Library publishes no limit, which the deploy issue correctly reads as "not a licence to hammer a nonprofit's service." The sharpest case is MusicBrainz (`coilyco-bridge/deploy#470`) at ~1 req/sec published. Two concurrent turns exceed it. There is currently no way to write a guardfile that does not. ## What would be enough A per-wrap bucket would cover every case in the batch: ```kdl wrap ward mcp musicbrainz { base-url "musicbrainz.org/ws/2" rate-limit 1/1s // requests to the upstream, process-wide } ``` Serialising at the outbound client rather than rejecting the caller is probably the right behaviour — a queued tool call is slower, a 503 is a failed turn. Per-grant granularity is not needed for this batch. ## What I verified Read against `mcp-beaver` `main` and `cli-guard v0.131.0` on 2026-08-13. I did not attempt an implementation. ## Related - `coilyco-bridge/deploy#465` — the epic carrying the requirement - `coilyco-bridge/deploy#468` — Open Library, shipped without it - `coilyco-bridge/deploy#470` — MusicBrainz, where this binds hardest

coilyco-ops referenced this issue from a commit

2026-08-14 05:24:08 +00:00

feat: add an outbound rate limit for public-good upstreams

coilyco-ops closed this issue

2026-08-14 05:24:08 +00:00

coilyco-ops commented

2026-08-14 05:38:48 +00:00

Author

Member

Landed in 18463c7.

rate-limit "1/1s"

Per-server and process-wide, matching the exposure you described: the pod has one IP, so the upstream sees one caller whose request rate is the sum of a whole community's curiosity. Per-grant granularity not offered, per your note that the batch does not need it - and because an upstream publishes one limit, not one per endpoint.

It waits rather than rejecting, which was your call and is the right one. The wait is bounded for free by the request deadline from #49, so a call that would queue past it fails with a stated timeout instead of holding a slot indefinitely. That interaction is tested.

Two placement choices that differ from the sketch, both deliberate.

Stated as a sibling of wrap, not inside it. Your sketch put rate-limit 1/1s in the wrap body. That body is opcore's frozen grammar and the umbra pin, and the runtime's half of this needs no grammar change - so it rides beside wrap like server-info and confirm. Costs one line of indentation, avoids a version bump on a shared dependency.
Applied at the tool handler, not by swapping opcore's HTTP client. Wrapping the client was the obvious implementation and it is a trap: RuntimeConfig.Client is nil-means-default, and that default is the redirect-guarding client - the one that refuses to follow a POST redirect, as seen in #55. Passing a limiter-wrapped client would have silently traded that guard away for a rate limit.

Grant-backed tools only. The info tool and withhold stubs reach no upstream, so charging them would throttle the fleet's liveness probe on behalf of a service it never calls. Placed inside any confirm gate, so a call waiting on a human does not hold a slot and a declined call has not spent one.

For the five servers: MusicBrainz is rate-limit "1/1s". Open Library has no published limit, so pick a courtesy rate rather than leaving it unset - the node is absent-means-unlimited, and this is a nonprofit's service.

Landed in 18463c7. ```kdl rate-limit "1/1s" ``` Per-server and process-wide, matching the exposure you described: the pod has one IP, so the upstream sees one caller whose request rate is the sum of a whole community's curiosity. Per-grant granularity not offered, per your note that the batch does not need it - and because an upstream publishes one limit, not one per endpoint. **It waits rather than rejecting**, which was your call and is the right one. The wait is bounded for free by the request deadline from #49, so a call that would queue past it fails with a stated timeout instead of holding a slot indefinitely. That interaction is tested. **Two placement choices that differ from the sketch, both deliberate.** 1. **Stated as a sibling of `wrap`, not inside it.** Your sketch put `rate-limit 1/1s` in the wrap body. That body is opcore's frozen grammar and the umbra pin, and the runtime's half of this needs no grammar change - so it rides beside `wrap` like `server-info` and `confirm`. Costs one line of indentation, avoids a version bump on a shared dependency. 2. **Applied at the tool handler, not by swapping opcore's HTTP client.** Wrapping the client was the obvious implementation and it is a trap: `RuntimeConfig.Client` is nil-means-default, and that default is the redirect-guarding client - the one that refuses to follow a POST redirect, as seen in #55. Passing a limiter-wrapped client would have silently traded that guard away for a rate limit. Grant-backed tools only. The info tool and `withhold` stubs reach no upstream, so charging them would throttle the fleet's liveness probe on behalf of a service it never calls. Placed inside any `confirm` gate, so a call waiting on a human does not hold a slot and a declined call has not spent one. For the five servers: MusicBrainz is `rate-limit "1/1s"`. Open Library has no published limit, so pick a courtesy rate rather than leaving it unset - the node is absent-means-unlimited, and this is a nonprofit's service.

coilyco-ops referenced this issue

2026-08-14 05:40:42 +00:00

First-class PDF processing: a large amount of authoritative reference material exists only as PDF #60

coilyco-ops referenced this issue

2026-08-14 05:40:48 +00:00

code-fleet-watch: policy findings #37