coilyco-gaming/sirens-echo

Watch

The skillpack loader discards every skill's description and inlines its body instead #971

New issue

Closed

opened 2026-08-18 19:11:46 +00:00 by coilyco-ops · 1 comment

coilyco-ops commented

2026-08-18 19:11:46 +00:00

Member

Implementation issue for what #968's evaluation turned up. #968 asked whether progressive disclosure works. It half works, and the half that does not is one function inverting the skill contract.

The defect

stripFrontmatter (internal/community/skillpack.go:265) returns remainder[end+5:], everything after the closing ---. The frontmatter block is dropped. loadSkills then writes that body straight into the prompt with no condition on it:

if isReferencePath(path) && !inlineAlways(text) { ... defer to read_skill ... }
fmt.Fprintf(&output, "\n## Source: %s\n\n%s\n", slashed, body)

For every entrypoint (SKILL.md, COMPOSED.md) the loader therefore:

discards name and description, the fields that exist to be a cheap always-on index
inlines the body, the part meant to be fetched when relevant

That is the skill contract backwards. description is read nowhere in skillpack.go or prompt.go; the only hits in the package are Discord command descriptions and the guardfile skill.

The catalogue is not the problem

Every one of the 70 composed skills carries a description. check_skill.py:358 fails the build on an empty one and caps it at 500 bytes, and enforces name matching the directory. agent-compose hands over a contract-conforming tree and the loader ignores the contract.

Corroborating detail: skillIndex labels each deferred reference with firstHeading(body), an H1 scraped from the text. The one place a description is exactly the right string, the code reaches for a fallback instead.

Measured

Engineer bundle, 51 catalogue skills, cl100k_base:

entrypoints inlined today - 28,714 tokens (128,334 bytes)
their names plus descriptions - 2,354 tokens (10,696 bytes)
headroom: 26,360 tokens, about 74% of the composed prompt

The reference tier from #859 is working and is not in scope here: 168,519 bytes deferred against 128,334 inlined, so it already carries 57% of the catalogue material.

What to build

Emit name plus description for an entrypoint instead of its body.
Add entrypoint paths to the read_skill index. The tool takes a repo-relative path already, so it needs no change.
Start using inline: true, on boundaries and capability. Nothing in the catalogue uses the flag today. #859's reason still holds: a model that has to choose to read its own boundaries may not.
Move whatever ValidateSystemPrompt anchors on out with the content it anchors on. This is the real work; steps 1 to 3 are small.

All four are harness changes. The catalogue needs no edit, which is the tell that it was never the catalogue's problem.

What not to do

Moving skill bodies into references/*.md was proposed and would work mechanically, since isReferencePath would defer them. It should still be refused:

.agents/composed/ is shared. agent-compose serves every harness from it, so gutting COMPOSED.md to satisfy one Go loader degrades every other consumer including native Claude Code skill install.
It inverts authoring and rollout. The defect is here, and fixing it upstream puts this repo's workaround in a repo that does not know this repo exists.
The index would get worse. coding-python would advertise itself as "Python" rather than the 200-character description already in its frontmatter.
check-skills blocks it, correctly. A COMPOSED.md reduced to a pointer is not a skill.

Sizing, and why this is not urgent

Do not sell this as a cost fix. Those 26K tokens sit in a byte-stable prefix that DeepSeek caches automatically. Measured over seven days of agent-proxy spans, sirens-echo/deepseek ran 99,292,714 input tokens against 42,894,848 cache reads, a 43.2% hit rate, priced 2.8e-09 against 1.4e-07 fresh. Optimizing tokens billed at a fiftieth is the wrong lever.

The two arguments that survive:

Attention. 51 full skill bodies competing for relevance every turn is a different problem from 51 descriptions, and caching does nothing for it. Argue this on recall and precision.
Cold prefill. Each image publish invalidates the prefix, so the first turn after every deploy pays it fresh, on a lane whose p95 community.turn was already 182.2s.

Neither is a reason to touch it before the 2026-08-19 stream.

Refs #859, #956, #964, #968, coilyco-bridge/deploy#932

Implementation issue for what #968's evaluation turned up. #968 asked whether progressive disclosure works. It half works, and the half that does not is one function inverting the skill contract. ## The defect `stripFrontmatter` (`internal/community/skillpack.go:265`) returns `remainder[end+5:]`, everything after the closing `---`. The frontmatter block is dropped. `loadSkills` then writes that body straight into the prompt with no condition on it: ```go if isReferencePath(path) && !inlineAlways(text) { ... defer to read_skill ... } fmt.Fprintf(&output, "\n## Source: %s\n\n%s\n", slashed, body) ``` For every entrypoint (`SKILL.md`, `COMPOSED.md`) the loader therefore: * **discards** `name` and `description`, the fields that exist to be a cheap always-on index * **inlines** the body, the part meant to be fetched when relevant That is the skill contract backwards. `description` is read nowhere in `skillpack.go` or `prompt.go`; the only hits in the package are Discord command descriptions and the guardfile skill. ## The catalogue is not the problem Every one of the **70** composed skills carries a `description`. `check_skill.py:358` fails the build on an empty one and caps it at 500 bytes, and enforces `name` matching the directory. agent-compose hands over a contract-conforming tree and the loader ignores the contract. Corroborating detail: `skillIndex` labels each deferred reference with `firstHeading(body)`, an H1 scraped from the text. The one place a description is exactly the right string, the code reaches for a fallback instead. ## Measured Engineer bundle, 51 catalogue skills, `cl100k_base`: * entrypoints inlined today - **28,714 tokens** (128,334 bytes) * their names plus descriptions - **2,354 tokens** (10,696 bytes) * **headroom: 26,360 tokens**, about 74% of the composed prompt The reference tier from #859 is working and is not in scope here: 168,519 bytes deferred against 128,334 inlined, so it already carries 57% of the catalogue material. ## What to build 1. Emit `name` plus `description` for an entrypoint instead of its body. 2. Add entrypoint paths to the `read_skill` index. The tool takes a repo-relative path already, so it needs no change. 3. Start using `inline: true`, on boundaries and capability. Nothing in the catalogue uses the flag today. #859's reason still holds: a model that has to choose to read its own boundaries may not. 4. Move whatever `ValidateSystemPrompt` anchors on out with the content it anchors on. This is the real work; steps 1 to 3 are small. All four are harness changes. **The catalogue needs no edit**, which is the tell that it was never the catalogue's problem. ## What not to do Moving skill bodies into `references/*.md` was proposed and would work mechanically, since `isReferencePath` would defer them. It should still be refused: * `.agents/composed/` is shared. agent-compose serves every harness from it, so gutting `COMPOSED.md` to satisfy one Go loader degrades every other consumer including native Claude Code skill install. * It inverts authoring and rollout. The defect is here, and fixing it upstream puts this repo's workaround in a repo that does not know this repo exists. * The index would get worse. `coding-python` would advertise itself as "Python" rather than the 200-character description already in its frontmatter. * `check-skills` blocks it, correctly. A `COMPOSED.md` reduced to a pointer is not a skill. ## Sizing, and why this is not urgent Do not sell this as a cost fix. Those 26K tokens sit in a byte-stable prefix that DeepSeek caches automatically. Measured over seven days of agent-proxy spans, `sirens-echo/deepseek` ran 99,292,714 input tokens against 42,894,848 cache reads, a **43.2% hit rate**, priced 2.8e-09 against 1.4e-07 fresh. Optimizing tokens billed at a fiftieth is the wrong lever. The two arguments that survive: * **Attention.** 51 full skill bodies competing for relevance every turn is a different problem from 51 descriptions, and caching does nothing for it. Argue this on recall and precision. * **Cold prefill.** Each image publish invalidates the prefix, so the first turn after every deploy pays it fresh, on a lane whose p95 `community.turn` was already 182.2s. Neither is a reason to touch it before the 2026-08-19 stream. Refs #859, #956, #964, #968, coilyco-bridge/deploy#932

coilyco-ops added the

role/engineer

label

2026-08-18 19:11:46 +00:00

coilyco-ops referenced this issue

2026-08-18 19:12:03 +00:00

Evaluate if skills based progressive disclosure is actually working #968

coilyco-ops commented

2026-08-18 19:16:53 +00:00

Author

Member

Correcting step 3 of this issue. I wrote that nothing uses inline: true and that a body-level split is when the flag would start earning its keep. Both halves are wrong, and found while closing #966 and #967.

What is actually true

frontmatterFlag (skillpack.go:247) accepts two spellings:

return strings.TrimSpace(value) == "always" || strings.TrimSpace(value) == "true"

The convention in the tree is inline: always, not inline: true, which is why a grep for the latter came back empty. Six local skill-root references use it today:

coilyco-general/references/capability.md
sirens-echo-knowledge/references/capability.md
sirens-echo-knowledge/references/boundaries.md
coilyco-org/references/organizations.md
sirens-dowel/references/site-work.md
sirens-dowel/references/dowel-provenance.md

That is close to exactly the set #859 predicted would have to stay inline: both capability docs and boundaries. The mechanism is not unproven and it is not waiting on this issue. It is in use and it is doing the job it was built for.

What survives of step 3

The gap is narrower and better defined than I described it:

the composed catalogue uses the flag nowhere. Zero of the .agents/composed/*/references/*.md files carry it, against six in the local roots. Whether that is correct or an oversight is worth a look on its own, since the catalogue is where boundary-* and role doctrine live.
step 3 is therefore a question, not a task. When entrypoints stop being inlined, which of the 51 bodies must not become optional? The local roots answer that question per file already. The catalogue has never had to.
spell it inline: always in anything built here, to match the six that exist rather than introducing a second accepted spelling into the tree.

The measured numbers in the issue body are unaffected: they were taken over the composed catalogue, where the flag genuinely has no users, so the 128,334 / 168,519 / 26,360 figures all stand.

The same correction applies to my comments on #968 and to coilysiren/inbox#380, which carry the same wrong claim.

Correcting step 3 of this issue. I wrote that nothing uses `inline: true` and that a body-level split is when the flag would start earning its keep. Both halves are wrong, and found while closing #966 and #967. ## What is actually true `frontmatterFlag` (`skillpack.go:247`) accepts **two** spellings: ```go return strings.TrimSpace(value) == "always" || strings.TrimSpace(value) == "true" ``` The convention in the tree is `inline: always`, not `inline: true`, which is why a grep for the latter came back empty. Six local skill-root references use it today: * `coilyco-general/references/capability.md` * `sirens-echo-knowledge/references/capability.md` * `sirens-echo-knowledge/references/boundaries.md` * `coilyco-org/references/organizations.md` * `sirens-dowel/references/site-work.md` * `sirens-dowel/references/dowel-provenance.md` That is close to exactly the set #859 predicted would have to stay inline: both capability docs and boundaries. The mechanism is not unproven and it is not waiting on this issue. It is in use and it is doing the job it was built for. ## What survives of step 3 The gap is narrower and better defined than I described it: * **the composed catalogue uses the flag nowhere.** Zero of the `.agents/composed/*/references/*.md` files carry it, against six in the local roots. Whether that is correct or an oversight is worth a look on its own, since the catalogue is where `boundary-*` and role doctrine live. * **step 3 is therefore a question, not a task.** When entrypoints stop being inlined, which of the 51 bodies must not become optional? The local roots answer that question per file already. The catalogue has never had to. * **spell it `inline: always`** in anything built here, to match the six that exist rather than introducing a second accepted spelling into the tree. The measured numbers in the issue body are unaffected: they were taken over the composed catalogue, where the flag genuinely has no users, so the 128,334 / 168,519 / 26,360 figures all stand. The same correction applies to my comments on #968 and to coilysiren/inbox#380, which carry the same wrong claim.

coilyco-ops referenced this issue from a commit

2026-08-19 00:51:31 +00:00

fix(skills): drop a trigger convention this harness does not have

coilyco-ops referenced this issue

2026-08-19 00:51:45 +00:00

feat(skills): give Moxn and Temporal each a skill their own people can PR #999

coilyco-ops referenced this issue from a commit

2026-08-19 01:15:51 +00:00

WIP fix(skills): restore the skill contract, entrypoints index rather than inline

coilyco-ops referenced this issue from a commit

2026-08-19 01:24:59 +00:00

test(skills): rewrite the three tests that pinned the old contract