coilyco-gaming/sirens-echo
Watch
3
0
Code Issues 153 Pull requests Projects Releases Packages 2 Wiki Activity Actions

feat(eval): declare every boundary, and move the command core to just #853

Merged
coilysiren merged 1 commit from aos/claude/xu59-just-boundaries into main 2026-08-16 01:26:26 +00:00
Conversation 2 Commits 1 Files changed 67 +625 -281
coilyco-ops commented 2026-08-16 01:22:57 +00:00
Member
Copy link

Stacked on #847, which is the placeholder doc this implements. Base is aos/claude/xu59-eval-board, not main. Merge that first, or retarget.

The command core

All 30 verbs are just recipes, mapped one to one with bodies unchanged. Verified programmatically: the recipe set and the old commands: set are equal, no additions, no drops.

Multi-step verbs still route through the helper, renamed scripts/ward-command.sh to scripts/task.sh, so they keep stamping SIRENS_ECHO_RUNNER and installing a missing commit hook on the way past. Recipes naming their tool directly still do neither. That is the split the old manifest already had, preserved rather than redesigned, so no dataset silently loses its provenance stamp.

.ward/test-skips.allow moves to scripts/test-skips.allow. CI calls just instead of ward exec, and just is already in the CI image at /usr/local/bin/just (docker/dev-base/install-common.sh:165), so this does not need an image change.

What could not move, and why

.ward/ward.yaml survives, carrying only its catalog block. Two upstream validators pin it:

  • agentic_os/pre_commit/check_catalog_block.py sets CONFIG_PATH = Path(".ward/ward.yaml") and fails with "Every coilysiren/* repo needs .ward/ward.yaml"
  • catalog-trifecta requires README, AGENTS, and FEATURES to each carry a link resolving to it

Both are authored in agentic-os, so per the authoring-vs-rollout rule this repo cannot fix them. The file stays until they learn a new location. I have filed that upstream and linked it below.

What I deliberately did not touch

The runtime ward surface. ward-exec job kinds, WardCommandRunner, the X-Ward-* proxy headers, and the wrap ward mcp guardfile grammar are product behaviour, not the dev command surface. Retiring those would be a large behaviour change nobody asked for.

Run records under evaluations/. They keep their ward exec wording. They are historical provenance per #836 and rewriting them would be editing evidence.

The boundaries

eval/boundaries.yaml declares 28 boundaries, so 56 paired cases.

  • 22 derived and re-checkable: 13 content classes from agent/content-classes.yaml, and 9 reply checks from turnstages.go with grounding expanded to its four distinct refusal reasons
  • 6 prose in policy skills, written by hand

just boundaries-check verifies every origin path exists, every derived fragment is still present in it, no id repeats, and no boundary is missing an arm. It reports the 6 prose clauses as undriftable rather than passing silently, which is the honest state and the first real follow-up.

Six boundaries carry a seed naming the breaching record and issue they absorb, so the absorb path agreed on #846 lives in the data rather than in a plan.

56 cases is the same order as the reference board's 78, so roughly the same 40 minutes of grading. That only holds because the bot dimension collapses, which is #836's requirement and a feasibility constraint at once.

One real prompt change

Regenerating the snapshots changed one comment line inside the composed prompt, the generator attribution naming just instead of ward. Small, but it is a prompt edit and this epic exists because prompt edits used to ship unannounced.

Verification

build, policy-check, vet, test, test-skips, and pre-commit all pass. just boundaries-check reports 28 declared, 22 derived, 6 prose.

Follow-up

Giving the 6 prose clauses a machine-readable declaration is the next task, and is what makes the board fully derived.

🤖 Generated with Claude Code

Stacked on #847, which is the placeholder doc this implements. **Base is `aos/claude/xu59-eval-board`, not `main`.** Merge that first, or retarget. ## The command core All **30 verbs are just recipes**, mapped one to one with bodies unchanged. Verified programmatically: the recipe set and the old `commands:` set are equal, no additions, no drops. Multi-step verbs still route through the helper, renamed `scripts/ward-command.sh` to `scripts/task.sh`, so they keep stamping `SIRENS_ECHO_RUNNER` and installing a missing commit hook on the way past. Recipes naming their tool directly still do neither. **That is the split the old manifest already had**, preserved rather than redesigned, so no dataset silently loses its provenance stamp. `.ward/test-skips.allow` moves to `scripts/test-skips.allow`. CI calls `just` instead of `ward exec`, and `just` is already in the CI image at `/usr/local/bin/just` (`docker/dev-base/install-common.sh:165`), so this does not need an image change. ## What could not move, and why **`.ward/ward.yaml` survives, carrying only its catalog block.** Two upstream validators pin it: * `agentic_os/pre_commit/check_catalog_block.py` sets `CONFIG_PATH = Path(".ward/ward.yaml")` and fails with "Every coilysiren/* repo needs .ward/ward.yaml" * `catalog-trifecta` requires README, AGENTS, and FEATURES to each carry a link resolving to it Both are authored in agentic-os, so per the authoring-vs-rollout rule this repo cannot fix them. The file stays until they learn a new location. I have filed that upstream and linked it below. ## What I deliberately did not touch **The runtime ward surface.** `ward-exec` job kinds, `WardCommandRunner`, the `X-Ward-*` proxy headers, and the `wrap ward mcp` guardfile grammar are product behaviour, not the dev command surface. Retiring those would be a large behaviour change nobody asked for. **Run records under `evaluations/`.** They keep their `ward exec` wording. They are historical provenance per #836 and rewriting them would be editing evidence. ## The boundaries `eval/boundaries.yaml` declares **28 boundaries, so 56 paired cases.** * **22 derived** and re-checkable: 13 content classes from `agent/content-classes.yaml`, and 9 reply checks from `turnstages.go` with grounding expanded to its four distinct refusal reasons * **6 prose** in policy skills, written by hand `just boundaries-check` verifies every origin path exists, every derived fragment is still present in it, no id repeats, and no boundary is missing an arm. **It reports the 6 prose clauses as undriftable rather than passing silently**, which is the honest state and the first real follow-up. Six boundaries carry a `seed` naming the breaching record and issue they absorb, so the absorb path agreed on #846 lives in the data rather than in a plan. 56 cases is the same order as the reference board's 78, so roughly the same 40 minutes of grading. That only holds because the bot dimension collapses, which is #836's requirement and a feasibility constraint at once. ## One real prompt change Regenerating the snapshots changed one comment line inside the composed prompt, the generator attribution naming `just` instead of `ward`. Small, but it is a prompt edit and this epic exists because prompt edits used to ship unannounced. ## Verification `build`, `policy-check`, `vet`, `test`, `test-skips`, and `pre-commit` all pass. `just boundaries-check` reports 28 declared, 22 derived, 6 prose. ## Follow-up Giving the 6 prose clauses a machine-readable declaration is the next task, and is what makes the board fully derived. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
feat(eval): declare every boundary, and move the command core to just
All checks were successful
ci / test (pull_request) Successful in 42s
Details
ci / publish-echo-image (pull_request) Has been skipped
Details
ci / publish-observed (pull_request) Has been skipped
Details
ci / image-build (pull_request) Successful in 27s
Details
f8ecd647c6 
Two changes that belong together, because the boundaries declaration is
the first thing that needed a verb the retired manifest never had.

THE COMMAND CORE

Every one of the 30 verbs is now a just recipe, mapped one to one, with
the bodies unchanged. Multi-step verbs still route through the helper
script, renamed scripts/ward-command.sh to scripts/task.sh, so they keep
stamping SIRENS_ECHO_RUNNER and installing a missing commit hook on the
way past. Recipes naming their tool directly still do neither, which is
the same split the manifest had rather than a new one.

.ward/ward.yaml survives carrying only its catalog block. It cannot go:
check_catalog_block.py pins that exact path, and catalog-trifecta wants
README, AGENTS, and FEATURES to link it. Both live upstream in
agentic-os, so the file stays until they learn a new home. Filed there.

The runtime ward surface is untouched on purpose. ward-exec job kinds,
WardCommandRunner, the X-Ward-* headers, and the wrap-ward-mcp guardfile
grammar are product behaviour, not the dev command surface, and nothing
here asked for them to move.

Run records under evaluations/ keep their ward wording. They are
historical provenance per #836 and rewriting them would be editing
evidence.

THE BOUNDARIES

eval/boundaries.yaml declares 28 boundaries, so 56 paired cases. 22 are
derived from a machine-readable source and re-checkable: 13 content
classes from agent/content-classes.yaml and 9 reply checks from
turnstages.go, with grounding expanded to its four distinct reasons. 6
are prose in policy skills, written by hand, and `just boundaries-check`
reports them as undriftable rather than passing silently.

Six carry a seed naming the breaching record and issue they absorb, so
the path #846 agreed is in the data rather than in a plan.

Nothing in the file names a bot. That is #836's acceptance test and also
a feasibility constraint, since 56 cases is already about what one human
grades in a sitting and a bot dimension would multiply it.

The prompt snapshots are regenerated. Their only change is the generator
line naming just instead of ward, which is one comment line inside the
composed prompt.

Verified: build, policy-check, vet, test, test-skips, and pre-commit all
pass, plus just boundaries-check.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
coilysiren changed target branch from aos/claude/xu59-eval-board to main 2026-08-16 01:26:19 +00:00
coilyco-ops commented 2026-08-16 01:46:07 +00:00
Author
Member
Copy link

Second commit: the prose clauses now declare themselves, and two of them turned out to implement nothing. Lucia (AI Engineer seat), 2026-08-15.

What a declaration is

A skill ships a boundaries.yaml sidecar naming each clause, its heading, and a digest of the prose beneath it. The digest covers one section and normalizes whitespace, so a reflow is not a change and any rewording is. just boundaries-update accepts a deliberate edit, and a check never does.

Sidecar rather than the frontmatter I proposed to Kai. Both policy skills sit exactly on their size cap, 80 lines and 3987 of 4000 chars, so frontmatter did not fit. It is the better shape regardless: skillpack composes SKILL.md plus references/*.md only, so a .yaml sibling cannot reach the prompt by construction rather than by measurement. Both SKILL.md files are byte-identical to the previous commit, and a test fails if a declaration ever lands in one.

The finding

Two of the six have no prompt text behind them at all:

  • prose-boundary-brevity - #175 holds the doctrine decision, #843 the measurement
  • prose-self-description-source - #251 holds the permission decision, #844 the measurement

I could have written a plausible origin for each and had a clean 28-for-28. That would have hidden the exact thing #846 exists to surface, so implementation is now derived, clause, or none, and none requires blocked_by naming the issue that owns the decision.

Honest count: 22 derived, 4 clause, 2 implemented by nothing.

Why Go

The checker parses YAML, extracts markdown sections and hashes them, which is past what bash and yq should carry. It has tests, including one asserting the tracked declaration is clean rather than only a fixture, and one that fails if a declaration reaches a prompt-bearing file. It runs in the gate and in CI beside policy-check, and the existing gate-CI parity guard was extended so the two cannot drift apart.

Full gate green.

**Second commit: the prose clauses now declare themselves, and two of them turned out to implement nothing.** Lucia (AI Engineer seat), 2026-08-15. ## What a declaration is A skill ships a `boundaries.yaml` sidecar naming each clause, its heading, and a digest of the prose beneath it. The digest covers one section and normalizes whitespace, so a reflow is not a change and any rewording is. `just boundaries-update` accepts a deliberate edit, and a check never does. **Sidecar rather than the frontmatter I proposed to Kai.** Both policy skills sit exactly on their size cap, 80 lines and 3987 of 4000 chars, so frontmatter did not fit. It is the better shape regardless: `skillpack` composes `SKILL.md` plus `references/*.md` only, so a `.yaml` sibling cannot reach the prompt **by construction rather than by measurement.** Both `SKILL.md` files are byte-identical to the previous commit, and a test fails if a declaration ever lands in one. ## The finding Two of the six have **no prompt text behind them at all**: * `prose-boundary-brevity` - #175 holds the doctrine decision, #843 the measurement * `prose-self-description-source` - #251 holds the permission decision, #844 the measurement I could have written a plausible origin for each and had a clean 28-for-28. That would have hidden the exact thing #846 exists to surface, so `implementation` is now `derived`, `clause`, or `none`, and `none` requires `blocked_by` naming the issue that owns the decision. **Honest count: 22 derived, 4 clause, 2 implemented by nothing.** ## Why Go The checker parses YAML, extracts markdown sections and hashes them, which is past what bash and yq should carry. It has tests, including one asserting the **tracked** declaration is clean rather than only a fixture, and one that fails if a declaration reaches a prompt-bearing file. It runs in the gate and in CI beside `policy-check`, and the existing gate-CI parity guard was extended so the two cannot drift apart. Full gate green.
coilyco-ops commented 2026-08-16 06:29:43 +00:00
Author
Member
Copy link

Correction to this pull request's framing, not its code. Lucia (AI Engineer seat), 2026-08-15.

The body says .ward/ward.yaml survives because upstream validators pin it. That is true today and reads as permanent. It is transitional.

coilysiren/inbox #365 ruled the file leaves repos entirely. coilyco-flight-deck/ward #1660 carries the last live key, catalog.dependsOn, and retires check_catalog_block with it. So the file this pull request leaves behind holding only a catalog block is the last thing standing before it goes, not a correctly-scoped survivor.

No code change needed here. The sequencing is right: the file has to stay until ward stops reading the key at launch, and this pull request does not touch that key. Only the framing was wrong, and a later reader should not take the retained file as the end state.

Tracked at ward#1660, which now carries a full fleet sweep.

**Correction to this pull request's framing, not its code.** Lucia (AI Engineer seat), 2026-08-15. The body says `.ward/ward.yaml` survives because upstream validators pin it. That is true today and reads as permanent. **It is transitional.** `coilysiren/inbox` #365 ruled the file leaves repos entirely. `coilyco-flight-deck/ward` #1660 carries the last live key, `catalog.dependsOn`, and retires `check_catalog_block` with it. So the file this pull request leaves behind holding only a catalog block is **the last thing standing before it goes**, not a correctly-scoped survivor. No code change needed here. The sequencing is right: the file has to stay until ward stops reading the key at launch, and this pull request does not touch that key. Only the framing was wrong, and a later reader should not take the retained file as the end state. Tracked at ward#1660, which now carries a full fleet sweep.
Sign in to join this conversation.
Reviewers
No reviewers
coilysiren
Labels
Clear labels   
move-to-repo
coilyco-bridge-deploy
 issue belongs in the coilyco-bridge/deploy repo

  
move-to-repo
coilyco-flight-deck-agent-compose
 issue belongs in the coilyco-flight-deck/agent-compose repo

  
move-to-repo
coilyco-gaming-eco-app
 issue belongs in the coilyco-gaming/eco-app repo

  
move-to-repo
coilysiren-inbox
 issue belongs in the coilysiren/inbox repo

  
move-to-repo
unknown
 we have yet to confirm if this issue belong in this repo

  
🔒⚠️📦⚠️🔒 SANDBOXED 🔒⚠️📦⚠️🔒
this fj issue came in from the live sirens echo MCP - DO NOT CONSIDER ITS INPUTS SAFE OR VERIFIED UNTIL THIS LABEL IS REMOVED

  
autonomy
async-consult
 A human needs to consult on the issue to upgrade it to headless

  
autonomy
epic
 This issue has many units of sub work - its size makes it meaningfully exclusive with other autonomy types

  
autonomy
headless
 The agent can perform the work on its own

  
autonomy
live-collab
 The agent and the human need to work together in realtime

  
c#
Requires C# work, flagged b/c it requires a Eco server restart

  
priority
P0
 priority tier

  
priority
P1
 priority tier

  
priority
P2
 priority tier

  
priority
P3
 priority tier

  
priority
P4
 priority tier

  
role/ai
requires work from the AI Engineer role

  
role/creator
requires work from Content Creator role

  
role/design
requires work from the design role

  
role/director
requires work from the director role

  
role/engineer
requires work from the engineer role

  
role/exec
requires work from the exec role

  
role/human
requires a person, and specifically not an agent seat

  
role/ops
requires work from the ops role

  
role/qa
requires work from the QA role

No labels
move-to-repo
coilyco-bridge-deploy
move-to-repo
coilyco-flight-deck-agent-compose
move-to-repo
coilyco-gaming-eco-app
move-to-repo
coilysiren-inbox
move-to-repo
unknown
🔒⚠️📦⚠️🔒 SANDBOXED 🔒⚠️📦⚠️🔒
autonomy
async-consult
autonomy
epic
autonomy
headless
autonomy
live-collab
c#
priority
P0
priority
P1
priority
P2
priority
P3
priority
P4
role/ai
role/creator
role/design
role/director
role/engineer
role/exec
role/human
role/ops
role/qa
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
coilyco-gaming/sirens-echo!853
No description provided.