Store behavior evaluations as YAML #98

New issue

Closed

opened 2026-07-25 12:45:34 +00:00 by coilyco-ops · 1 comment

coilyco-ops commented

2026-07-25 12:45:34 +00:00

Member

Goal

Make YAML the canonical encoding for behavior evaluation packs and scored baseline records.

Acceptance criteria

Hand-migrate every tracked evaluations/latest/*.json record to readable .yaml without changing any raw response, score, evidence sentence, total, verdict, model, or provenance field.
Replace scored-result JSON decoding with strict YAML decoding and add deterministic YAML encoding for future records.
Change agent-compose evaluation --format from json to yaml while retaining Markdown as the default review view.
Remove evaluation-specific JSON wording and tests, and reject tracked JSON baselines.
Update the feature inventory and evaluation documentation in the same commit.
Run the full Ward test suite.

Source

Kai requested the evaluation storage and writer migration after #96 completed the current Codex baseline set.

## Goal Make YAML the canonical encoding for behavior evaluation packs and scored baseline records. ## Acceptance criteria * Hand-migrate every tracked `evaluations/latest/*.json` record to readable `.yaml` without changing any raw response, score, evidence sentence, total, verdict, model, or provenance field. * Replace scored-result JSON decoding with strict YAML decoding and add deterministic YAML encoding for future records. * Change `agent-compose evaluation --format` from `json` to `yaml` while retaining Markdown as the default review view. * Remove evaluation-specific JSON wording and tests, and reject tracked JSON baselines. * Update the feature inventory and evaluation documentation in the same commit. * Run the full Ward test suite. ## Source Kai requested the evaluation storage and writer migration after #96 completed the current Codex baseline set.

coilyco-ops referenced this issue from a commit

2026-07-25 12:59:28 +00:00

feat(evaluation): store baselines as yaml

coilyco-ops closed this issue

2026-07-25 12:59:28 +00:00

coilyco-ops commented

2026-07-25 13:00:04 +00:00

Author

Member

Landed on canonical main in commit e17ec30.

Hand-migrated all ten scored Codex baselines from JSON to YAML without changing their logical records.
Switched the evaluation pack CLI to --format yaml and removed legacy JSON output.
Added strict YAML scored-result decoding plus validated deterministic YAML encoding for future baselines.
Added regression coverage for CLI YAML output, JSON rejection, multiline block scalars, deterministic result encoding, current-pack validation, and the absence of tracked JSON results.
Updated README, the feature inventory, and the evaluation contract.
Full ward exec test passed, including all Go tests, palette build, documentation gates, trufflehog, whitespace checks, and catalog hooks.

Follow-up #97 now names evaluations/latest/community-codex.yaml for the first Community baseline.

Landed on canonical `main` in commit `e17ec30`. * Hand-migrated all ten scored Codex baselines from JSON to YAML without changing their logical records. * Switched the evaluation pack CLI to `--format yaml` and removed legacy JSON output. * Added strict YAML scored-result decoding plus validated deterministic YAML encoding for future baselines. * Added regression coverage for CLI YAML output, JSON rejection, multiline block scalars, deterministic result encoding, current-pack validation, and the absence of tracked JSON results. * Updated README, the feature inventory, and the evaluation contract. * Full `ward exec test` passed, including all Go tests, palette build, documentation gates, trufflehog, whitespace checks, and catalog hooks. Follow-up #97 now names `evaluations/latest/community-codex.yaml` for the first Community baseline.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

coilyco-flight-deck/agent-compose#98

No description provided.

Rows
Columns