coilyco-flight-deck/agent-compose
coilyco-flight-deck/agent-compose
Watch
2
0
Code Issues 20 Pull requests Projects Releases 161 Packages Wiki Activity Actions

fix: make Designer frontier evaluation green #177

Merged
coilysiren merged 2 commits from issue-163-designer-evals into main 2026-08-02 19:18:43 +00:00
Conversation 0 Commits 2 Files changed 10 +466 -631
coilyco-ops commented 2026-08-02 11:38:34 +00:00
Member
Copy link

Closes #163

Summary

  • Tightens the Designer evidence, completion, authority, and Content Manager boundaries.
  • Records the exact current Design frontier and OSS responses with an independent rubric review.
  • Adds fail-closed v2 evidence for empty model content with a terminal finish reason.

Evidence

  • Current Design pack digest: sha256:244dfaad780638fb91f9a6aa7249c8ae7f1cb810a10199a817216d8a7d5eb33c
  • Frontier: all 6 cases pass, 47/48.
  • OSS: 5 of 6 cases pass. The empty length-limited completion response remains an explicit 0/8 failure.
  • The other seven role result files remain byte-unchanged.

Validation

  • ward exec evaluation-scorecard-check
  • ward exec test
  • ward exec smoke
Closes #163 Summary * Tightens the Designer evidence, completion, authority, and Content Manager boundaries. * Records the exact current Design frontier and OSS responses with an independent rubric review. * Adds fail-closed v2 evidence for empty model content with a terminal finish reason. Evidence * Current Design pack digest: sha256:244dfaad780638fb91f9a6aa7249c8ae7f1cb810a10199a817216d8a7d5eb33c * Frontier: all 6 cases pass, 47/48. * OSS: 5 of 6 cases pass. The empty length-limited completion response remains an explicit 0/8 failure. * The other seven role result files remain byte-unchanged. Validation * ward exec evaluation-scorecard-check * ward exec test * ward exec smoke
Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Sign in to join this conversation.
Reviewers
No reviewers
coilysiren
Labels
Clear labels
  
autonomy
async-consult
 A human needs to consult on the issue to upgrade it to headless

  
autonomy
epic
 This issue has many units of sub work - its size makes it meaningfully exclusive with other autonomy types

  
autonomy
headless
 The agent can perform the work on its own

  
autonomy
live-collab
 The agent and the human need to work together in realtime

  
coherence-core
Core review set for the warded control plane coherence milestone. These issues form the release spine; adjacent milestone issues are stretch or supporting work.

  
priority
P0
 priority tier

  
priority
P1
 priority tier

  
priority
P2
 priority tier

  
priority
P3
 priority tier

  
priority
P4
 priority tier

  
qa-fixture
Disposable issue admitted to the bounded Ward QA verification lane.

  
role/ai
requires work from the AI Engineer role

  
role/creator
requires work from Content Creator role

  
role/design
requires work from the design role

  
role/director
requires work from the director role

  
role/engineer
requires work from the engineer role

  
role/exec
requires work from the exec role

  
role/human
requires a person, and specifically not an agent seat

  
role/ops
requires work from the ops role

  
role/qa
requires work from the QA role

No labels
autonomy
async-consult
autonomy
epic
autonomy
headless
autonomy
live-collab
coherence-core
priority
P0
priority
P1
priority
P2
priority
P3
priority
P4
qa-fixture
role/ai
role/creator
role/design
role/director
role/engineer
role/exec
role/human
role/ops
role/qa
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
coilyco-flight-deck/agent-compose!177
No description provided.