test(prompt): derive the shipped profile set from disk #974

Merged

coilysiren merged 1 commit from task/derive-shipped-profiles into main

2026-08-18 19:45:00 +00:00

coilyco-ops commented

2026-08-18 19:41:00 +00:00

Member

Same shape as #972 and coilyco-bridge/deploy#673, one layer in: two inventories
in prompt_test.go named agents/echo and agents/deep by hand, and both fail
silently when a third profile arrives.

What was hand-listed

TestThePrincipalUserIDNeverReachesThePrompt iterated a literal pair. That
check exists because the user ID was rendered into every prompt, inside a
paragraph telling the model not to rely on it, and Deep recited it (issue 166).
A profile added later would simply not have been checked.

promptBudgets is the other. Its numbers are ratchets and stay
hand-set — a raise is a decision someone writes down in
docs/sirens-echo-prompt.md. But the set is not a decision, and a rendered
prompt with no entry grows unwatched forever. That is the silent half of a
ratchet.

What changed

shippedProfiles globs agents/*/<leaf> and fails when it matches nothing,
so an empty glob cannot quietly turn either check into a no-op. The principal
check iterates it.

TestEveryRenderedPromptHasABudget is new and asserts completeness rather
than a number, so the two halves stay separate: the set is derived, the
numbers stay written down.

Verified by making it fail

Not trusted, run. A synthetic agents/probe-lane/rendered/prompt.txt:

--- FAIL: TestEveryRenderedPromptHasABudget
    agents/probe-lane/rendered/prompt.txt has no promptBudgets entry,
    so nothing watches it grow

and green again once removed. Full go test ./... passes, pre-commit clean.

Not done here, deliberately

agent/rendered/roles/*.bundle.txt has no budget of any kind, and
cmd/sirens-echo-prompt/main.go says the role bundle size is "reported and
never gated", verbatim. That is the same silent shape one surface over, and it
is the surface that actually moved: the engineer bundle is ~160KB against Echo's
19,042-byte ratchet.

I did not add it, because picking the first number is a call about what a
composed lane may cost rather than a derivation. It wants its own change and an
owner. Related context in coilysiren/inbox#380.

Same shape as #972 and coilyco-bridge/deploy#673, one layer in: two inventories in `prompt_test.go` named `agents/echo` and `agents/deep` by hand, and both fail **silently** when a third profile arrives. ## What was hand-listed **`TestThePrincipalUserIDNeverReachesThePrompt`** iterated a literal pair. That check exists because the user ID was rendered into every prompt, inside a paragraph telling the model not to rely on it, and Deep recited it (issue 166). A profile added later would simply not have been checked. **`promptBudgets`** is the other. Its numbers are ratchets and **stay hand-set** — a raise is a decision someone writes down in `docs/sirens-echo-prompt.md`. But the *set* is not a decision, and a rendered prompt with no entry grows unwatched forever. That is the silent half of a ratchet. ## What changed `shippedProfiles` globs `agents/*/<leaf>` and **fails when it matches nothing**, so an empty glob cannot quietly turn either check into a no-op. The principal check iterates it. `TestEveryRenderedPromptHasABudget` is new and asserts **completeness rather than a number**, so the two halves stay separate: the set is derived, the numbers stay written down. ## Verified by making it fail Not trusted, run. A synthetic `agents/probe-lane/rendered/prompt.txt`: ``` --- FAIL: TestEveryRenderedPromptHasABudget agents/probe-lane/rendered/prompt.txt has no promptBudgets entry, so nothing watches it grow ``` and green again once removed. Full `go test ./...` passes, `pre-commit` clean. ## Not done here, deliberately `agent/rendered/roles/*.bundle.txt` has **no budget of any kind**, and `cmd/sirens-echo-prompt/main.go` says the role bundle size is "reported and never gated", verbatim. That is the same silent shape one surface over, and it is the surface that actually moved: the engineer bundle is ~160KB against Echo's 19,042-byte ratchet. I did not add it, because picking the first number is a call about what a composed lane may cost rather than a derivation. It wants its own change and an owner. Related context in coilysiren/inbox#380.

coilyco-ops added 1 commit

2026-08-18 19:41:00 +00:00

test(prompt): derive the shipped profile set from disk

ci / image-build (pull_request) Successful in 36s

Details

ci / test (pull_request) Successful in 1m1s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

85eedfa56c

Two inventories in prompt_test.go named agents/echo and agents/deep by hand,
and both fail silently when a third profile arrives.

TestThePrincipalUserIDNeverReachesThePrompt iterated a literal pair. That check
exists because the user ID was rendered into every prompt inside a paragraph
telling the model not to rely on it, and Deep recited it (issue 166). A profile
added later would not have been checked, and nothing would have said so.

promptBudgets is the other. The numbers there are ratchets and stay hand-set: a
raise is a decision someone writes down in docs/sirens-echo-prompt.md. The SET
is not a decision, and a rendered prompt with no entry grows unwatched forever,
which is the silent half of a ratchet.

shippedProfiles globs agents/*/<leaf> and fails when it matches nothing, so an
empty glob cannot quietly turn either check into a no-op. The principal check
iterates it. TestEveryRenderedPromptHasABudget is new and asserts completeness
rather than a number, so the two halves stay separate.

Verified by making it fail rather than trusting it: a synthetic
agents/probe-lane/rendered/prompt.txt produces

    agents/probe-lane/rendered/prompt.txt has no promptBudgets entry,
    so nothing watches it grow

and the suite goes green again once it is removed.

NOT DONE HERE. agent/rendered/roles/*.bundle.txt has no budget of any kind, and
cmd/sirens-echo-prompt says the role bundle size is "reported and never gated".
That is the same silent shape one surface over, but picking the first number is
a call about what a composed lane may cost rather than a derivation, so it wants
its own change and an owner.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

coilyco-ops requested review from coilysiren

2026-08-18 19:41:00 +00:00