Refresh communication-boundary evaluation evidence #220

New issue

Closed

opened 2026-08-06 07:57:31 +00:00 by coilyco-ops · 0 comments

coilyco-ops commented

2026-08-06 07:57:31 +00:00

Member

Outcome

Refresh the Agent Compose Core Roster evaluation baseline for the communication-ownership correction in #210.

Use the AI Engineer role with the canonical eval-role-comms method. Generate evidence from source revision 739baab61b1e5bcad7fab7da9c821bcb9580a206 on branch issue-210.

Required evaluation

Generate the current Core Roster packs through ward exec evaluation-packs.
Run every active frontier case in a fresh isolated driver session.
Preserve the existing #178 recommendation-deferral cases.
Exercise the new Ops rollout ledger, Engineer implementation checkpoint, QA verdict record, and Director decision record cases.
Treat communication strategy by a non-Content role and over-deferral of an authorized factual work record as hard failures.
Use an independent frontier-high QA reviewer. Preserve raw responses, finish reasons, retries, criterion evidence, exact source revision, and pack digest.
Refresh evaluations/latest/ and docs/evaluation-scores.md.
Run ward exec test, commit the evidence to issue-210, and update PR #221.

Boundary

The source contract, evaluator rubric, matrices, docs, and evaluator skill are already implemented and repository-tested at the named revision. The current caller-assigned Engineer session cannot activate the AI-owned evaluation method or serve as its independent QA reviewer.

## Outcome Refresh the Agent Compose Core Roster evaluation baseline for the communication-ownership correction in #210. Use the AI Engineer role with the canonical `eval-role-comms` method. Generate evidence from source revision `739baab61b1e5bcad7fab7da9c821bcb9580a206` on branch `issue-210`. ## Required evaluation * Generate the current Core Roster packs through `ward exec evaluation-packs`. * Run every active frontier case in a fresh isolated driver session. * Preserve the existing #178 recommendation-deferral cases. * Exercise the new Ops rollout ledger, Engineer implementation checkpoint, QA verdict record, and Director decision record cases. * Treat communication strategy by a non-Content role and over-deferral of an authorized factual work record as hard failures. * Use an independent frontier-high QA reviewer. Preserve raw responses, finish reasons, retries, criterion evidence, exact source revision, and pack digest. * Refresh `evaluations/latest/` and `docs/evaluation-scores.md`. * Run `ward exec test`, commit the evidence to `issue-210`, and update PR #221. ## Boundary The source contract, evaluator rubric, matrices, docs, and evaluator skill are already implemented and repository-tested at the named revision. The current caller-assigned Engineer session cannot activate the AI-owned evaluation method or serve as its independent QA reviewer.

coilyco-ops referenced this issue

2026-08-06 07:57:43 +00:00

Allow role-owned factual communications #221

coilyco-ops referenced this issue

2026-08-06 07:58:09 +00:00

Allow role-owned status updates and mechanical communications #210

coilyco-ops referenced this issue from a commit

2026-08-06 08:29:24 +00:00

test: refresh communication evaluation evidence, closes #220