Classify failing PR checks before dispatching repair engineers #1109

New issue

Closed

opened 2026-07-11 02:04:15 +00:00 by coilyco-ops · 2 comments

coilyco-ops commented

2026-07-11 02:04:15 +00:00

Member

Problem

PR repair workers currently fall into a single vague "CI/test repair" path when a pull request check is failing. That makes the director repeatedly dispatch engineers without first classifying the failure mode.

The repair path needs to distinguish at least four cases:

CI parity gap: Forgejo runs a job that the engineer cannot faithfully reproduce locally.
Main is red: the same check or equivalent local suite fails on current origin/main, so the PR is not the cause.
Merge queue churn: the PR is otherwise valid, but main moved or Forgejo mergeability/check state needs a branch refresh.
PR-introduced failure: the PR branch fails a local or CI check that passes on origin/main.

Desired behavior

Teach the PR repair workflow to classify the failure before it dispatches or loops. A failing PR check should produce a concrete next action instead of another blind engineer launch.

Acceptance criteria:

The repair path records or prints which bucket it selected: ci-parity-gap, main-red, merge-queue-churn, or pr-regression.
For a CI parity gap, ward should point at the missing local ward exec <job> mirror or another concrete parity issue instead of guessing from unrelated local tests.
For main-red, ward should stop blaming the PR and surface a main-failure incident/follow-up path.
For merge-queue churn, ward should update/rebase/refresh the branch once, then avoid repeated blind dispatches if the same condition persists.
For PR-introduced failures, ward should keep the current engineer-repair behavior.
Add focused tests around the classifier or dispatch decision so this does not rely on prompt discipline alone.

Notes

This came from the 2026-07-11 director audit after the agent described "CI / test repair loops." Kai clarified that CI-only parity failures and main-red failures are distinct from normal PR repair and from merge queue churn. The goal is to make ward encode that distinction at the dispatch chokepoint, not merely ask engineers to remember it.

## Problem PR repair workers currently fall into a single vague "CI/test repair" path when a pull request check is failing. That makes the director repeatedly dispatch engineers without first classifying the failure mode. The repair path needs to distinguish at least four cases: * CI parity gap: Forgejo runs a job that the engineer cannot faithfully reproduce locally. * Main is red: the same check or equivalent local suite fails on current `origin/main`, so the PR is not the cause. * Merge queue churn: the PR is otherwise valid, but `main` moved or Forgejo mergeability/check state needs a branch refresh. * PR-introduced failure: the PR branch fails a local or CI check that passes on `origin/main`. ## Desired behavior Teach the PR repair workflow to classify the failure before it dispatches or loops. A failing PR check should produce a concrete next action instead of another blind engineer launch. Acceptance criteria: * The repair path records or prints which bucket it selected: `ci-parity-gap`, `main-red`, `merge-queue-churn`, or `pr-regression`. * For a CI parity gap, ward should point at the missing local `ward exec <job>` mirror or another concrete parity issue instead of guessing from unrelated local tests. * For main-red, ward should stop blaming the PR and surface a main-failure incident/follow-up path. * For merge-queue churn, ward should update/rebase/refresh the branch once, then avoid repeated blind dispatches if the same condition persists. * For PR-introduced failures, ward should keep the current engineer-repair behavior. * Add focused tests around the classifier or dispatch decision so this does not rely on prompt discipline alone. ## Notes This came from the 2026-07-11 director audit after the agent described "CI / test repair loops." Kai clarified that CI-only parity failures and main-red failures are distinct from normal PR repair and from merge queue churn. The goal is to make ward encode that distinction at the dispatch chokepoint, not merely ask engineers to remember it.

coilyco-ops added the

headless

labels

2026-07-11 02:04:15 +00:00

coilyco-ops commented

2026-07-11 02:04:33 +00:00

Author

Member

WARD-RESERVATION: held 🔒

reservation details

Holder: container engineer-codex-ward-1109 on host KAI-DESKTOP-TOWER.

Reserved by ward agent --harness codex (reserved 2026-07-11T02:04:33Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (1h TTL). --force overrides.

Do not comment on or edit this issue to steer the run while it is reserved. The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a new issue, dispatched fresh. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).

run seed context — what this run is carrying (ward#609)

Resolved: coilyco-flight-deck/ward#1109 · branch issue-1109 · harness codex · workflow pull-requests-and-merge
Run: engineer-codex-ward-1109 · ward v0.605.0 · dispatched 2026-07-11T02:04:33Z
Comment thread: 0 included in the pre-flight read, 0 stripped (ward's own automated comments).

Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.605.0).

— Codex, via ward agent

WARD-RESERVATION: held 🔒 <details><summary>reservation details</summary> Holder: container `engineer-codex-ward-1109` on host `KAI-DESKTOP-TOWER`. Reserved by `ward agent --harness codex` (reserved 2026-07-11T02:04:33Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (1h TTL). `--force` overrides. **Do not comment on or edit this issue to steer the run while it is reserved.** The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a **new issue, dispatched fresh**. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494). <details><summary>run seed context — what this run is carrying (ward#609)</summary> - **Resolved:** `coilyco-flight-deck/ward#1109` · branch `issue-1109` · harness `codex` · workflow `pull-requests-and-merge` - **Run:** `engineer-codex-ward-1109` · ward `v0.605.0` · dispatched `2026-07-11T02:04:33Z` - **Comment thread:** 0 included in the pre-flight read, 0 stripped (ward's own automated comments). Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.605.0). </details> </details>  — Codex, via `ward agent`

coilyco-ops referenced this issue from a pull request that will close it,

2026-07-11 02:25:35 +00:00

Classify failing PR checks before dispatching repair engineers #1110

coilyco-ops commented

2026-07-11 02:26:25 +00:00

Author

Member

WARD-OUTCOME: submitted

details

workflow: pull-request-and-merge; review summary: review gate intentionally skipped because the temporary ward default pending brokered QA

implementation felt direct once the PR status and repo-config surfaces were folded into one classifier.

confidence: high

surprise: the repo-level lint hook required the classifier helper to stay small.

follow-ups: none.

WARD-OUTCOME: submitted <details><summary>details</summary> workflow: pull-request-and-merge; review summary: review gate intentionally skipped because the temporary ward default pending brokered QA implementation felt direct once the PR status and repo-config surfaces were folded into one classifier. confidence: high surprise: the repo-level lint hook required the classifier helper to stay small. follow-ups: none. </details>

coilyco-ops closed this issue

2026-07-11 02:27:02 +00:00