Persist remaining scored Codex role baselines #96

New issue

Closed

opened 2026-07-25 12:28:10 +00:00 by coilyco-ops · 1 comment

coilyco-ops commented

2026-07-25 12:28:10 +00:00

Member

Goal

Complete repository-backed coverage for the evaluated Codex roles that are still missing from evaluations/latest/.

Acceptance criteria

Add validated latest records for Director, Advisor, Ops, PM, Designer, Social, Sales, and Customer Success.
Preserve the accepted raw responses, criterion evidence, totals, verdicts, model identities, and immutable source revisions from issues #84 and #91.
Retain the newer Engineer and QA records from #80 unchanged.
Document the resulting coverage boundary, including that Community has no accepted scored baseline yet.
Run the full Ward test suite.

Source

Kai requested continuation of the agent-compose evaluations after #95 landed the first two scored records.

## Goal Complete repository-backed coverage for the evaluated Codex roles that are still missing from `evaluations/latest/`. ## Acceptance criteria * Add validated latest records for Director, Advisor, Ops, PM, Designer, Social, Sales, and Customer Success. * Preserve the accepted raw responses, criterion evidence, totals, verdicts, model identities, and immutable source revisions from issues #84 and #91. * Retain the newer Engineer and QA records from #80 unchanged. * Document the resulting coverage boundary, including that Community has no accepted scored baseline yet. * Run the full Ward test suite. ## Source Kai requested continuation of the agent-compose evaluations after #95 landed the first two scored records.

coilyco-ops referenced this issue from a commit

2026-07-25 12:38:10 +00:00

feat(evaluation): persist remaining scored baselines

coilyco-ops closed this issue

2026-07-25 12:38:10 +00:00

coilyco-ops commented

2026-07-25 12:38:30 +00:00

Author

Member

Landed on canonical main in commit 64b4368.

Added validated Codex records for Director, Advisor, Ops, PM, Designer, Social, Sales, and Customer Success.
Preserved accepted raw responses, criterion evidence, totals, verdicts, model identities, issue provenance, and immutable source revisions from #84 and #91.
Retained the newer Engineer and QA records from #80 unchanged.
Documented ten-role Codex coverage and the absence of an accepted Community baseline.
Full ward exec test passed, including generated-pack result validation, the palette build, all catalog hooks, and the offline trufflehog scan.

The stored results keep the honest boundary visible. Every frontier case in this slice passes the strict rule. The recorded OSS cases remain mixed or failing and were not normalized into green results.

Landed on canonical `main` in commit `64b4368`. * Added validated Codex records for Director, Advisor, Ops, PM, Designer, Social, Sales, and Customer Success. * Preserved accepted raw responses, criterion evidence, totals, verdicts, model identities, issue provenance, and immutable source revisions from #84 and #91. * Retained the newer Engineer and QA records from #80 unchanged. * Documented ten-role Codex coverage and the absence of an accepted Community baseline. * Full `ward exec test` passed, including generated-pack result validation, the palette build, all catalog hooks, and the offline trufflehog scan. The stored results keep the honest boundary visible. Every frontier case in this slice passes the strict rule. The recorded OSS cases remain mixed or failing and were not normalized into green results.

coilyco-ops referenced this issue

2026-07-25 12:38:43 +00:00

Score the initial Community Codex behavior baseline #97

coilyco-ops referenced this issue

2026-07-25 12:45:34 +00:00

Store behavior evaluations as YAML #98