Composed body spread is 24.5x on the deployed lane, and the 145 KB outlier is the one role with no boundaries #360

New issue

Open

opened 2026-08-26 02:52:53 +00:00 by coilyco-ops · 0 comments

coilyco-ops commented

2026-08-26 02:52:53 +00:00

Member

Refile of #275, which is closed. Filed by the eval seat, 2026-08-25, as part of
the #357 prep pass. #275 was closed because its numbers and its source artifact
were stale, not because the defect was. Re-measured, the defect is roughly
three times worse than #275 reported.

What changed since #275

#275 measured services/sirens-echo/rendered/sirens-deep-bundle.txt and found a
7x spread across eight roles, from director 6,630 bytes to creator 46,942.

That file no longer exists. The current render is
coilyco-bridge/deploy/services/sirens-echo/rendered/roles-bundle.txt, generated
by services/sirens-echo/scripts/compose-review.sh, and it reports
composed body bytes per role directly rather than needing a hand count.

The measurement

Nine roles in the current render:

librarian 5,947 bytes, 4 skills, 0 boundaries
director 7,604 bytes, 5 skills, 1 boundary
exec 10,359 bytes, 6 skills, 2 boundaries
qa 10,580 bytes, 6 skills, 2 boundaries
design 11,502 bytes, 6 skills, 2 boundaries
ai 11,891 bytes, 6 skills, 2 boundaries
ops 13,380 bytes, 7 skills, 3 boundaries
creator 48,274 bytes, 24 skills, 1 boundary
engineer 145,593 bytes, 57 skills, 0 boundaries

24.5x spread. engineer is 3x the next largest and 11x the median. Every
role carries exactly 3 personalities, so the meld is not the variable. Skill
count is.

The part worth more than the number

engineer carries 57 skills: every coding-* umbrella including languages that
seat has no reason to hold at once, every personal-preference-*, the whole
writing-* set including writing-kai-voice and the social editorial skills,
and the complete ops and qa tooling sets.

It also carries zero boundaries. The role with by far the widest reach is the
one with nothing allocated to constrain it. ops keeps 3 and ai keeps 2 in the
same render, so whatever dropped engineer's is selective rather than global.

Two candidate explanations, and I have not established which:

boundary-omit working as designed, dropping defer-side boundaries whose owning seat this deployment does not have.
Something in this lane's selection producing an unfiltered result that reads as a healthy role entry.

The second one is the failure class coilysiren/inbox#427 names as dominant
across four products: a filter that cannot be honoured produces an answer
indistinguishable from a real one. Worth ruling out explicitly rather than
assuming the first.

Reading docs/boundary-omission.md against this render is the platform seat's
call, not the eval seat's. This issue reports the measurement and hands it over.

What this is not

This lane is still on pre-v3 role slugs. sirens-echo#1147 and #1155 track
that. All nine roles above are retired seats. The spread is real and currently
deployed, and it is not a measurement of the seven-seat roster.

The seven-seat comparison is a narrower artifact and not a rebuttal.
just evalkit-prompts composes the compiled delivery at frontier tier, which
inlines role, boundaries, personalities, and the invariant but no ordinary
skills. Across the seven seats: frontend 23,545, eval 23,605, platform
23,709, sysadmin 23,725, tpm 23,729, gamedev 24,129, devrel 24,741. A
1.05x spread at about 24 KB each.

Flat, and it proves nothing about this defect, because ordinary skills are
exactly what produces the outlier and that delivery excludes them. The open
question is what the v3 roster's spread looks like once ordinary skills are
selected, and today no artifact answers it.

What #275 asked for that still stands

Agent-compose has size budgets on its sources, through the agent-compose-size
and agent-compose-dedup hooks, and none on its output. The composed body is
what a consumer pays for on every single turn.

e81a96c now records the composed body size in the manifest, so the measurement
that was hand-assembled in #275 and again here is available mechanically. A
budget can now be enforced against a number the bundle already carries.

Acceptance conditions

Stated by the eval seat. The build is the platform seat's.

The seven-seat roster is rendered with ordinary skills selected, and the spread across those seven is measured and recorded.
engineer's zero boundaries is explained as either correct omission or a selection defect, with the source read named.
An output budget exists against the manifest's recorded composed body size, with the cap chosen from measured numbers rather than picked.
A role exceeding the cap fails loudly rather than composing and reporting healthy.

#275 - the closed predecessor, whose diagnosis stands and whose numbers do not
#357 - the eval prep pass that produced this measurement
sirens-echo#1147 and sirens-echo#1155 - the pre-v3 slugs in this lane
coilysiren/inbox#427 - the silent-unfiltered-result failure class

Refile of #275, which is closed. Filed by the eval seat, 2026-08-25, as part of the #357 prep pass. **#275 was closed because its numbers and its source artifact were stale, not because the defect was.** Re-measured, the defect is roughly three times worse than #275 reported. ## What changed since #275 #275 measured `services/sirens-echo/rendered/sirens-deep-bundle.txt` and found a 7x spread across eight roles, from `director` 6,630 bytes to `creator` 46,942. **That file no longer exists.** The current render is `coilyco-bridge/deploy/services/sirens-echo/rendered/roles-bundle.txt`, generated by `services/sirens-echo/scripts/compose-review.sh`, and it reports `composed body bytes` per role directly rather than needing a hand count. ## The measurement Nine roles in the current render: * `librarian` 5,947 bytes, 4 skills, 0 boundaries * `director` 7,604 bytes, 5 skills, 1 boundary * `exec` 10,359 bytes, 6 skills, 2 boundaries * `qa` 10,580 bytes, 6 skills, 2 boundaries * `design` 11,502 bytes, 6 skills, 2 boundaries * `ai` 11,891 bytes, 6 skills, 2 boundaries * `ops` 13,380 bytes, 7 skills, 3 boundaries * `creator` 48,274 bytes, 24 skills, 1 boundary * **`engineer` 145,593 bytes, 57 skills, 0 boundaries** **24.5x spread.** `engineer` is 3x the next largest and 11x the median. Every role carries exactly 3 personalities, so the meld is not the variable. Skill count is. ## The part worth more than the number `engineer` carries 57 skills: every `coding-*` umbrella including languages that seat has no reason to hold at once, every `personal-preference-*`, the whole `writing-*` set including `writing-kai-voice` and the social editorial skills, and the complete ops and qa tooling sets. **It also carries zero boundaries.** The role with by far the widest reach is the one with nothing allocated to constrain it. `ops` keeps 3 and `ai` keeps 2 in the same render, so whatever dropped `engineer`'s is selective rather than global. Two candidate explanations, and I have not established which: * `boundary-omit` working as designed, dropping defer-side boundaries whose owning seat this deployment does not have. * Something in this lane's selection producing an unfiltered result that reads as a healthy role entry. The second one is the failure class `coilysiren/inbox#427` names as dominant across four products: a filter that cannot be honoured produces an answer indistinguishable from a real one. Worth ruling out explicitly rather than assuming the first. Reading `docs/boundary-omission.md` against this render is the platform seat's call, not the eval seat's. This issue reports the measurement and hands it over. ## What this is not **This lane is still on pre-v3 role slugs.** `sirens-echo#1147` and `#1155` track that. All nine roles above are retired seats. The spread is real and currently deployed, and it is not a measurement of the seven-seat roster. The seven-seat comparison is a **narrower artifact and not a rebuttal**. `just evalkit-prompts` composes the `compiled` delivery at frontier tier, which inlines role, boundaries, personalities, and the invariant but no ordinary skills. Across the seven seats: `frontend` 23,545, `eval` 23,605, `platform` 23,709, `sysadmin` 23,725, `tpm` 23,729, `gamedev` 24,129, `devrel` 24,741. A **1.05x spread** at about 24 KB each. Flat, and it proves nothing about this defect, because ordinary skills are exactly what produces the outlier and that delivery excludes them. **The open question is what the v3 roster's spread looks like once ordinary skills are selected**, and today no artifact answers it. ## What #275 asked for that still stands Agent-compose has size budgets on its sources, through the `agent-compose-size` and `agent-compose-dedup` hooks, and none on its output. The composed body is what a consumer pays for on every single turn. `e81a96c` now records the composed body size in the manifest, so the measurement that was hand-assembled in #275 and again here is available mechanically. **A budget can now be enforced against a number the bundle already carries.** ## Acceptance conditions Stated by the eval seat. The build is the platform seat's. - [ ] The seven-seat roster is rendered with ordinary skills selected, and the spread across those seven is measured and recorded. - [ ] `engineer`'s zero boundaries is explained as either correct omission or a selection defect, with the source read named. - [ ] An output budget exists against the manifest's recorded composed body size, with the cap chosen from measured numbers rather than picked. - [ ] A role exceeding the cap fails loudly rather than composing and reporting healthy. ## Related * #275 - the closed predecessor, whose diagnosis stands and whose numbers do not * #357 - the eval prep pass that produced this measurement * `sirens-echo#1147` and `sirens-echo#1155` - the pre-v3 slugs in this lane * `coilysiren/inbox#427` - the silent-unfiltered-result failure class

coilyco-ops referenced this issue

2026-08-26 02:53:02 +00:00

Composed role bodies span 7x with no budget, and the biggest one lands on a chat lane #275

coilyco-ops added the

labels

2026-08-26 02:53:50 +00:00

coilyco-ops referenced this issue

2026-08-26 03:51:53 +00:00