docs(evals): current state, plus the rest of the stack #308

Merged

coilysiren merged 2 commits from aos/claude/eval-context into main

2026-08-18 17:26:23 +00:00

coilyco-ops commented

2026-08-18 17:22:53 +00:00

Member

Supersedes #307, which was cut before main moved.

The page hedged that sirens-echo "shares the boundary declaration shape today, not the dataset". True this morning, not now: coilyco-gaming/sirens-echo#957 made board-deep emit aos-eval's dataset shape with a profile and a contract test, so the pairing rule has one implementation across the two runners.

It also had no way out. A closing section links the shared layer, the other consumer, and the writeup.

Paid for by cutting what a reader gets from the tool instead of the page: the evalkit-* verb list is just --list, and the export payload's field enumeration is in the exporter's own output. The non-obvious parts stay, including why evalkit-check runs from a script rather than a pre-commit hook.

120 of 120 lines, docs/*.md at 39. pre-commit run --all-files passes.

🤖 Generated with Claude Code

Supersedes #307, which was cut before `main` moved. The page hedged that sirens-echo "shares the boundary declaration shape today, not the dataset". True this morning, not now: coilyco-gaming/sirens-echo#957 made `board-deep` emit aos-eval's dataset shape with a profile and a contract test, so the pairing rule has one implementation across the two runners. It also had no way out. A closing section links the shared layer, the other consumer, and the writeup. Paid for by cutting what a reader gets from the tool instead of the page: the `evalkit-*` verb list is `just --list`, and the export payload's field enumeration is in the exporter's own output. The non-obvious parts stay, including why `evalkit-check` runs from a script rather than a pre-commit hook. 120 of 120 lines, `docs/*.md` at 39. `pre-commit run --all-files` passes. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

coilyco-ops added 1 commit

2026-08-18 17:22:53 +00:00

docs(evals): current state, plus the rest of the stack

ci / test (pull_request) Successful in 1m18s

Details

12df680e16

Two things this page could not tell a reader.

It hedged that sirens-echo "shares the boundary declaration shape today, not the
dataset", which was true this morning and is not now: that repo's board-deep
emits aos-eval's dataset shape, its profile declares its column, and a contract
test pins both sides. The pairing rule has one implementation across the two
runners rather than two.

And it had no way out. A reader landing here could not reach the shared layer, the
other consumer, or the writeup. A closing section links all three.

Paid for the additions by cutting what a reader can get from the tool instead of
the page: the evalkit verb list is `just --list`, and the export payload's field
enumeration is in the exporter's own output. The non-obvious parts stay, including
why evalkit-check runs from a script rather than a pre-commit hook.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

coilyco-ops requested review from coilysiren

2026-08-18 17:22:53 +00:00

coilyco-ops added 1 commit

2026-08-18 17:24:58 +00:00

Merge branch 'main' into aos/claude/eval-context

ci / test (pull_request) Successful in 1m2s

Details

1373521402

#307 landed the same sirens-echo correction this branch carried, reworded. Takes
this branch's wording, which reads "the pairing rule has one implementation across
both" rather than "the pairing rule below", since the rest-of-stack section now
sits below it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>