coilyco-flight-deck/mcp-beaver

Watch

Caveats serialized last are destroyed first by a truncating consumer, so every generated server can hand back a bounded result that looks complete #68

New issue

Closed

opened 2026-08-15 15:47:57 +00:00 by coilyco-ops · 1 comment

coilyco-ops commented

2026-08-15 15:47:57 +00:00

Member

Filed by Darren (director seat) at Kai's direction, 2026-08-15. Cross-cutting finding from coilyco-gaming/sirens-echo#449, being filed on every MCP server in the portfolio because the exposure is structural rather than specific to the server that surfaced it.

The failure, concretely

A community agent told a member:

Currently 0 wooden hull planks are listed for sale on the server. No active store shelves or priced trade history exists for this item.

The tool result it built that from carried its own caveat:

22405 older trades arrive as 3152 hourly rollups; party, item, store,
and unit-price views cover detailed rows only
No markets matched item='wooden hull plank' across 528 ledger rows

The surface said it searched 528 of 22,933 rows. The reply said none exists. Those are different claims and only the first is true. It cost a day of investigation into a query path that turned out to be fine.

The mechanism that generalizes

This is the part worth propagating, and it is arithmetic rather than model behaviour.

A consuming harness bounds tool results at a byte cap. The consumer measured in #449 does it as a head slice — keep the front, discard the tail, append a byte-count notice:

return result[:cut] + fmt.Sprintf(truncationNotice, cut, len(result)), cut, true

The server that surfaced this serializes warnings as its last JSON key. So whenever its response exceeds the cap, the caveats are the first thing removed, deterministically. The model receives rows with no caveat and answers as though the view were complete.

Two properties make this nastier than an ordinary truncation bug:

The notice is in bytes, not in meaning. The model learns "this was cut" and never learns "the search covered 528 of 22,933 rows". Only the second changes the answer.
The cap is per-consumer. In the observed case one profile capped at 8192 and another at 16384. The same query is honest on one and silently caveat-free on the other, and nothing in either log says so.

Measured in the sirens-echo consumer. Inferred for other consumers: head-slicing at a byte budget is the common shape, so a server cannot assume its trailing fields survive. That inference is the reason this belongs server-side. A server that keeps its response under the cap never depends on which consumer read it.

Why this repo carries the most leverage

mcp-beaver generates servers from policy files. Every server it emits inherits its response-shaping decisions, and the guarded servers already in the fleet came out of it. Fixing the property here fixes it for the generated fleet at once, rather than once per server.

The four invariants below are the ask. Two of them are already written up from the other direction in coilyco-gaming/eco-app#266 and #267, which are worth reading before implementing.

Coverage and caveat metadata serialize first, never last. A field that explains the bound of a result must not sit where a head slice removes it. This is the new finding and the one Kai had not seen before.
limit bounds every unbounded array, not one of them. eco-app#267 documents a tool returning ~45 KB at limit=1, because limit bounded one array of six. Responses that overrun a consumer cap are what makes rule 1 load-bearing in the first place.
Truncation always warns, naming shown-of-total. Silent truncation is prohibited. A count is not a caveat: "3152 of 22933" is actionable, "truncated" is not.
An unreadable dataset reports null, never zero. Absence of data and a measured zero must not serialize alike.

What to do here

Audit what the generator emits against the four rules, including field ordering in the serialized response.
Decide whether rules 1 and 3 can be enforced by the generator itself rather than left to each policy file. A generated envelope that always leads with coverage would make the property structural instead of advisory.
Record the decision, since a generator that cannot enforce it needs to say so where server authors will read it.

Acceptance

Every response shape the generator emits places coverage and caveat fields ahead of bulk payload.
No generated tool can return an unbounded array.
Truncation emits shown-of-total, and an unreadable dataset is distinguishable from an empty one.
If any rule is left to the server author rather than enforced, that is documented rather than implicit.

Source: coilyco-gaming/sirens-echo#449, including its seven-comment thread which establishes the head-slice mechanism, the per-profile cap difference, and why the consumer-side check is hard. The consumer-side half stays open there and is not what this issue asks for.

**Filed by Darren (director seat) at Kai's direction, 2026-08-15.** Cross-cutting finding from coilyco-gaming/sirens-echo#449, being filed on every MCP server in the portfolio because the exposure is structural rather than specific to the server that surfaced it. ## The failure, concretely A community agent told a member: > Currently 0 wooden hull planks are listed for sale on the server. No active store shelves or priced trade history exists for this item. The tool result it built that from carried its own caveat: ``` 22405 older trades arrive as 3152 hourly rollups; party, item, store, and unit-price views cover detailed rows only No markets matched item='wooden hull plank' across 528 ledger rows ``` The surface said it searched 528 of 22,933 rows. The reply said none exists. Those are different claims and only the first is true. It cost a day of investigation into a query path that turned out to be fine. ## The mechanism that generalizes This is the part worth propagating, and it is arithmetic rather than model behaviour. A consuming harness bounds tool results at a byte cap. The consumer measured in #449 does it as a **head slice** — keep the front, discard the tail, append a byte-count notice: ```go return result[:cut] + fmt.Sprintf(truncationNotice, cut, len(result)), cut, true ``` The server that surfaced this serializes `warnings` as its **last** JSON key. So whenever its response exceeds the cap, the caveats are the first thing removed, deterministically. The model receives rows with no caveat and answers as though the view were complete. Two properties make this nastier than an ordinary truncation bug: * **The notice is in bytes, not in meaning.** The model learns "this was cut" and never learns "the search covered 528 of 22,933 rows". Only the second changes the answer. * **The cap is per-consumer.** In the observed case one profile capped at 8192 and another at 16384. The same query is honest on one and silently caveat-free on the other, and nothing in either log says so. **Measured** in the sirens-echo consumer. **Inferred** for other consumers: head-slicing at a byte budget is the common shape, so a server cannot assume its trailing fields survive. That inference is the reason this belongs server-side. A server that keeps its response under the cap never depends on which consumer read it. ## Why this repo carries the most leverage mcp-beaver generates servers from policy files. Every server it emits inherits its response-shaping decisions, and the guarded servers already in the fleet came out of it. Fixing the property here fixes it for the generated fleet at once, rather than once per server. The four invariants below are the ask. Two of them are already written up from the other direction in coilyco-gaming/eco-app#266 and #267, which are worth reading before implementing. 1. **Coverage and caveat metadata serialize first, never last.** A field that explains the bound of a result must not sit where a head slice removes it. This is the new finding and the one Kai had not seen before. 2. **`limit` bounds every unbounded array, not one of them.** eco-app#267 documents a tool returning ~45 KB at `limit=1`, because `limit` bounded one array of six. Responses that overrun a consumer cap are what makes rule 1 load-bearing in the first place. 3. **Truncation always warns, naming shown-of-total.** Silent truncation is prohibited. A count is not a caveat: "3152 of 22933" is actionable, "truncated" is not. 4. **An unreadable dataset reports null, never zero.** Absence of data and a measured zero must not serialize alike. ## What to do here * Audit what the generator emits against the four rules, including field ordering in the serialized response. * Decide whether rules 1 and 3 can be enforced by the generator itself rather than left to each policy file. A generated envelope that always leads with coverage would make the property structural instead of advisory. * Record the decision, since a generator that cannot enforce it needs to say so where server authors will read it. ## Acceptance * Every response shape the generator emits places coverage and caveat fields ahead of bulk payload. * No generated tool can return an unbounded array. * Truncation emits shown-of-total, and an unreadable dataset is distinguishable from an empty one. * If any rule is left to the server author rather than enforced, that is documented rather than implicit. --- Source: coilyco-gaming/sirens-echo#449, including its seven-comment thread which establishes the head-slice mechanism, the per-profile cap difference, and why the consumer-side check is hard. The consumer-side half stays open there and is not what this issue asks for.

coilyco-ops referenced this issue from coilyco-flight-deck/node-stats-mcp

2026-08-15 15:48:55 +00:00

Audit tool responses for caveat-last ordering: a truncating consumer destroys trailing coverage fields first, so a bounded result arrives looking complete #22

coilyco-ops referenced this issue from coilyco-flight-deck/bluesky-mcp

2026-08-15 15:49:52 +00:00

Audit tool responses for caveat-last ordering: a truncating consumer destroys trailing coverage fields first, so a bounded result arrives looking complete #12

coilyco-ops referenced this issue from coilyco-flight-deck/lunch-money-k8s

2026-08-15 15:50:22 +00:00

Audit tool responses for caveat-last ordering: a truncating consumer destroys trailing coverage fields first, so a bounded result arrives looking complete #42

coilyco-ops referenced this issue from coilyco-gaming/steam-ops

2026-08-15 15:50:52 +00:00

Audit tool responses for caveat-last ordering: a truncating consumer destroys trailing coverage fields first, so a bounded result arrives looking complete #18

coilyco-ops added the

autonomy

headless

label

2026-08-15 15:51:23 +00:00

coilyco-ops referenced this issue from coilyco-gaming/sirens-echo

2026-08-15 15:52:38 +00:00

A tool result that says it saw 2 percent of the data becomes a reply that says none exists #449

coilyco-ops referenced this issue from coilyco-gaming/sirens-echo

2026-08-15 16:27:20 +00:00

Measure whether an absence claim ever coexists with an OK-outcome tool call naming the same item, which is the discriminator #449's fix depends on #811

coilyco-ops referenced this issue from coilyco-gaming/sirens-echo

2026-08-15 16:27:58 +00:00

A tool result that says it saw 2 percent of the data becomes a reply that says none exists #449

coilyco-ops added the

priority

role/engineer

labels

2026-08-17 01:14:24 +00:00

coilyco-ops referenced this issue from a commit

2026-08-17 02:31:23 +00:00

feat(instructions): let a guardfile say which dam you are at (#77)

coilyco-ops referenced this issue

2026-08-17 02:31:38 +00:00

Every generated server publishes the same instructions, so the handshake says nothing about which dam you are at #77

coilyco-ops referenced this issue from a commit

2026-08-17 02:37:28 +00:00

feat(result): lead every result with coverage so a head slice cannot cut it (#68)

coilyco-ops closed this issue

2026-08-17 02:37:28 +00:00

coilyco-ops commented

2026-08-17 02:37:47 +00:00

Author

Member

Built on main in e809ee1. Rules 1 and 3 are enforced by the generator rather than left to each policy file, which was the open question. Rules 2 and 4 partly cannot be, and that is recorded rather than implicit.

What the audit found

The generator emitted {"result": <upstream payload>} as structured content and the bare upstream body as text content. Two problems past the one filed:

Nothing in the envelope stated a bound at all, so there was no caveat field to serialize first or last. Rule 1 had nothing to order.
The text and structured halves disagreed, and the text half is the one a consumer slices. Leading only the structured half would have left the caveat exactly where it always got destroyed.

Enforced

Every grant-backed result is now {"coverage": {...}, "result": ...}, in that order, in both halves.

The mechanism is worth naming because it is the part that cannot rot: the envelope is a Go struct, not a map. encoding/json writes struct fields in declaration order and map keys in sorted order, so a struct makes the position a contract instead of an accident of whatever the next caveat field gets named. TestCoverageSerializesBeforeThePayload asserts the order and also that coverage starts within the first 64 bytes - "earlier than the payload" is not the requirement, surviving a head slice is.

Coverage states only what this runtime can honestly measure:

truncated - always present, always false, because nothing here truncates. The standing explicit claim is what lets a consumer attribute a short view to its own slicing rather than to the server.
bytes and over_budget - over_budget trips past 8192, the smaller of the two caps you measured. It does not mean this runtime cut anything; it means some consumer will, and the model may be reading a prefix.
items - every array in the payload, by name, with its length. This is where the eco-app#267 shape (45 KB at limit=1, because limit bounded one array of six) becomes visible on the first call rather than after an investigation.

It deliberately never reports a shown-of-total. The total lives upstream and this runtime never asked for it. Inventing one would be the same confident wrong answer the issue is about.

Not enforceable here, and now said out loud in `docs/DESIGN.md`

Rule 2, limit bounding every array. limit is a query parameter this runtime forwards; nothing here can make an upstream bound a second array it chose not to. What the runtime does instead is refuse to let that stay invisible - the unbounded array arrives with a named count and an over_budget flag, rather than as a response that merely feels large.
Rule 4, null versus zero. This runtime never synthesizes a zero: an unreadable upstream is an isError tool result and a decoded null stays null, both pinned by test. But an upstream that answers a failed read with 0 is reporting its own state, and no envelope can tell that apart from a measured zero.
Upstream-proxy mode passes the upstream's own CallToolResult through, since preserving that contract is the point of the mode. The envelope there is the upstream's to shape.
Fixed-shape tools (serve-ssm, mcp_beaver_info, withheld stubs) carry no coverage block. None can grow with upstream data, and a coverage block on a response that cannot be partial trains a reader to skim it.

Contract note for consumers

.result is unchanged in meaning and position within the object, so anything reading it keeps working. Two things did change: the tool outputSchema now requires coverage alongside result, and text content is the envelope rather than the bare upstream body. Anything parsing content[0].text as the upstream's own shape needs .result. That brings text into line with structured content, which has wrapped in {"result": ...} all along.

Read eco-app#266 and #267 before wiring the deploy-side half - they are the same property from the other direction.

Built on `main` in `e809ee1`. Rules 1 and 3 are enforced by the generator rather than left to each policy file, which was the open question. Rules 2 and 4 partly cannot be, and that is recorded rather than implicit. ## What the audit found The generator emitted `{"result": <upstream payload>}` as structured content and the **bare upstream body** as text content. Two problems past the one filed: * Nothing in the envelope stated a bound at all, so there was no caveat field to serialize first or last. Rule 1 had nothing to order. * The text and structured halves **disagreed**, and the text half is the one a consumer slices. Leading only the structured half would have left the caveat exactly where it always got destroyed. ## Enforced Every grant-backed result is now `{"coverage": {...}, "result": ...}`, in that order, in **both** halves. The mechanism is worth naming because it is the part that cannot rot: the envelope is a Go **struct**, not a map. `encoding/json` writes struct fields in declaration order and map keys in sorted order, so a struct makes the position a contract instead of an accident of whatever the next caveat field gets named. `TestCoverageSerializesBeforeThePayload` asserts the order and also that coverage starts within the first 64 bytes - "earlier than the payload" is not the requirement, surviving a head slice is. Coverage states only what this runtime can honestly measure: * `truncated` - always present, always false, because nothing here truncates. The standing explicit claim is what lets a consumer attribute a short view to its own slicing rather than to the server. * `bytes` and `over_budget` - `over_budget` trips past 8192, the smaller of the two caps you measured. It does not mean this runtime cut anything; it means some consumer will, and the model may be reading a prefix. * `items` - every array in the payload, by name, with its length. This is where the eco-app#267 shape (45 KB at `limit=1`, because `limit` bounded one array of six) becomes visible on the first call rather than after an investigation. It deliberately **never** reports a shown-of-total. The total lives upstream and this runtime never asked for it. Inventing one would be the same confident wrong answer the issue is about. ## Not enforceable here, and now said out loud in `docs/DESIGN.md` * **Rule 2, `limit` bounding every array.** `limit` is a query parameter this runtime forwards; nothing here can make an upstream bound a second array it chose not to. What the runtime does instead is refuse to let that stay invisible - the unbounded array arrives with a named count and an `over_budget` flag, rather than as a response that merely feels large. * **Rule 4, null versus zero.** This runtime never synthesizes a zero: an unreadable upstream is an `isError` tool result and a decoded `null` stays `null`, both pinned by test. But an upstream that answers a failed read with `0` is reporting its own state, and no envelope can tell that apart from a measured zero. * **Upstream-proxy mode** passes the upstream's own `CallToolResult` through, since preserving that contract is the point of the mode. The envelope there is the upstream's to shape. * **Fixed-shape tools** (`serve-ssm`, `mcp_beaver_info`, withheld stubs) carry no coverage block. None can grow with upstream data, and a coverage block on a response that cannot be partial trains a reader to skim it. ## Contract note for consumers `.result` is unchanged in meaning and position within the object, so anything reading it keeps working. Two things did change: the tool `outputSchema` now requires `coverage` alongside `result`, and **text content is the envelope rather than the bare upstream body**. Anything parsing `content[0].text` as the upstream's own shape needs `.result`. That brings text into line with structured content, which has wrapped in `{"result": ...}` all along. Read eco-app#266 and #267 before wiring the deploy-side half - they are the same property from the other direction.

coilyco-ops referenced this issue

2026-08-17 03:06:24 +00:00

First-class PDF processing: a large amount of authoritative reference material exists only as PDF #60

coilyco-ops referenced this issue from a commit

2026-08-19 09:08:59 +00:00

fix(coverage): count arrays nested under objects, so a GraphQL response stops reporting nothing

coilyco-ops referenced this issue from a commit

2026-08-19 09:34:36 +00:00

fix(coverage): state a bounded sql read's truncation where a caveat survives