feat(eval): declare the Dowel lane's boundaries and author its board #982

Merged

coilysiren merged 1 commit from aos/claude/dowel-board into main

2026-08-18 20:57:46 +00:00

coilyco-ops commented

2026-08-18 20:52:39 +00:00

Member

Dowel had no eval subject at all. agents/ held deep and echo, the verbs were eval-echo, eval-deep, and board-deep, and eval/boundaries.yaml named Dowel zero times. eval-deep is not a stand-in either: Deep composes creator, Dowel composes engineer, and Dowel adds the sirens-dowel skill root, so running it would report a clean result about a different agent.

Twelve boundaries, scoped to the lane

They carry role: dowel, which aos-eval already supports per boundary, so a board derives the lane it is measuring rather than every lane at once.

They go in the shared declaration rather than a file of their own because scripts/boundaries.sh hardcodes FILE=eval/boundaries.yaml. A separate file would not be drift-checked, and an origin nobody checks is exactly the thing that goes stale.

All twelve origins resolve against files this lane actually carries. just boundaries-check now reports 40 declared, 34 derived from source, 6 prose.

The shared content-* and reply-* boundaries still bind Dowel, because they come from the harness every lane runs. Four prose-* entries do not: they cite .agents/skills/sirens-echo-community, a root Dowel does not load. That is the reason this is a Dowel declaration rather than a copy of Deep's, and it is worth knowing before anyone points Deep's board at this lane.

One boundary was not exercisable, so I reshaped it

dowel-moxn-outage asked for the behaviour on a failed knowledge-base call. A prompt cannot make a credential expire, so the pair could never be authored honestly. It is now dowel-moxn-empty-result: an absent document reaches the same rule through a turn that can actually get there.

The board

agents/dowel/evaluations/board-slots.yaml, in aos-eval's dataset shape, so boundaries check and annotate --profile eval/aos-eval-profile.yaml read it with no adapter.

Only prompt is authored. id, boundary, half, pair_id, and target are the derived values, which is the split aos-eval boundaries derive documents: the target comes from the declaration and nothing invents a prompt.

output is empty on all 24. aos-eval holds no runner and no model client, so nothing in the grading layer fills it. Dowel is a live Discord lane rather than a fixture, so an output arrives from a real turn in #moxn-temporal, either through the lane's turn surface or by a human posting the prompt. That is the one gap between this board and a runnable one.

dowel-favorite-color-in is the board's validity check. It reproduces the failure this whole thread started from, so the profile orders it first among the Dowel groups, matching the reason no-invented-surface leads Deep's.

Verified rather than asserted

just boundaries-check - 40 declared, 34 derived from source, every Dowel origin resolving.
aos-eval boundaries check - 0 Dowel slots missing, 0 unpaired, 0 undeclared. It still exits 1 on the 56 shared slots that remain unauthored, which is the honest signal rather than a coverage percentage.
aos-eval validate --profile - 24 samples match the profile.
Repository hooks pass. The code-comments hook rejects YAML comments below the top header, so all commentary sits in the header blocks.

Not done here

The 56 shared slots are still unauthored, and the Deep board remains the 10-case pilot slice that closes none of them. That is the next slice and a bigger one.

Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com

Generated with Claude Code

Dowel had no eval subject at all. `agents/` held `deep` and `echo`, the verbs were `eval-echo`, `eval-deep`, and `board-deep`, and `eval/boundaries.yaml` named Dowel zero times. `eval-deep` is not a stand-in either: Deep composes `creator`, Dowel composes `engineer`, and Dowel adds the `sirens-dowel` skill root, so running it would report a clean result about a different agent. ## Twelve boundaries, scoped to the lane They carry `role: dowel`, which `aos-eval` already supports per boundary, so a board derives the lane it is measuring rather than every lane at once. They go in the shared declaration rather than a file of their own because `scripts/boundaries.sh` hardcodes `FILE=eval/boundaries.yaml`. A separate file would not be drift-checked, and an origin nobody checks is exactly the thing that goes stale. All twelve origins resolve against files this lane actually carries. `just boundaries-check` now reports **40 declared, 34 derived from source, 6 prose**. The shared `content-*` and `reply-*` boundaries still bind Dowel, because they come from the harness every lane runs. Four `prose-*` entries do not: they cite `.agents/skills/sirens-echo-community`, a root Dowel does not load. That is the reason this is a Dowel declaration rather than a copy of Deep's, and it is worth knowing before anyone points Deep's board at this lane. ## One boundary was not exercisable, so I reshaped it `dowel-moxn-outage` asked for the behaviour on a failed knowledge-base call. A prompt cannot make a credential expire, so the pair could never be authored honestly. It is now `dowel-moxn-empty-result`: an absent document reaches the same rule through a turn that can actually get there. ## The board `agents/dowel/evaluations/board-slots.yaml`, in aos-eval's dataset shape, so `boundaries check` and `annotate --profile eval/aos-eval-profile.yaml` read it with no adapter. Only `prompt` is authored. `id`, `boundary`, `half`, `pair_id`, and `target` are the derived values, which is the split `aos-eval boundaries derive` documents: the target comes from the declaration and nothing invents a prompt. **`output` is empty on all 24.** aos-eval holds no runner and no model client, so nothing in the grading layer fills it. Dowel is a live Discord lane rather than a fixture, so an output arrives from a real turn in `#moxn-temporal`, either through the lane's `turn` surface or by a human posting the prompt. That is the one gap between this board and a runnable one. `dowel-favorite-color-in` is the board's validity check. It reproduces the failure this whole thread started from, so the profile orders it first among the Dowel groups, matching the reason `no-invented-surface` leads Deep's. ## Verified rather than asserted * `just boundaries-check` - 40 declared, 34 derived from source, every Dowel origin resolving. * `aos-eval boundaries check` - **0** Dowel slots missing, **0** unpaired, **0** undeclared. It still exits 1 on the 56 shared slots that remain unauthored, which is the honest signal rather than a coverage percentage. * `aos-eval validate --profile` - 24 samples match the profile. * Repository hooks pass. The `code-comments` hook rejects YAML comments below the top header, so all commentary sits in the header blocks. ## Not done here The 56 shared slots are still unauthored, and the Deep board remains the 10-case pilot slice that closes none of them. That is the next slice and a bigger one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Generated with [Claude Code](https://claude.com/claude-code)

coilyco-ops added 1 commit

2026-08-18 20:52:39 +00:00

feat(eval): declare the Dowel lane's boundaries and author its board

ci / image-build (pull_request) Successful in 51s

Details

ci / test (pull_request) Successful in 1m14s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

95f6523af4

Dowel had no eval subject at all. agents/ held deep and echo, the verbs were
eval-echo, eval-deep, and board-deep, and eval/boundaries.yaml named Dowel
zero times. eval-deep is not a stand-in either: Deep composes creator, Dowel
composes engineer, and Dowel adds the sirens-dowel skill root, so running it
would report a clean result about a different agent.

Twelve boundaries, scoped with `role: dowel` so a board derives the lane it is
measuring rather than every lane at once. They go in the shared declaration
instead of a file of their own because scripts/boundaries.sh only drift-checks
eval/boundaries.yaml, and an origin nobody checks is the thing that goes stale.
All twelve origins resolve: `just boundaries-check` reports 40 declared, 34
derived from source.

The shared content-* and reply-* boundaries still bind Dowel, since they come
from the harness every lane runs. Four prose-* entries do not, because they
cite .agents/skills/sirens-echo-community and Dowel does not load that root.
That is why this is a Dowel declaration rather than a copy of Deep's.

dowel-moxn-empty-result was reshaped from a credential-outage pair. A prompt
cannot make a credential expire, so the pair was unexercisable as written. An
absent document reaches the same rule.

The board is aos-eval's dataset shape, so `boundaries check` and `annotate`
read it with no adapter. Only `prompt` is authored, which is the split the
deriver documents: id, boundary, half, pair_id, and target are derived values.
`output` is empty on all 24, because aos-eval holds no runner and Dowel is a
live lane rather than a fixture.

dowel-favorite-color-in reproduces the failure this board came from and is its
validity check, so the profile orders it first among the Dowel groups.

Verified: 0 dowel slots missing, 0 unpaired, 0 undeclared, 24 samples match
the profile, and the repository hooks pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

coilyco-ops requested review from coilysiren

2026-08-18 20:52:39 +00:00