coilyco-flight-deck/ward

Fork 0

issues with stale reservations should be in scope for director backlog automated burndown flow #978

New issue

Closed

opened 2026-07-10 08:58:04 +00:00 by coilysiren · 3 comments

coilysiren commented

2026-07-10 08:58:04 +00:00

Owner

The problem

when headless runs die, the director automated backlog flow just leaves them alone for forever b/c it thinks they are being worked on

Proposed change

backlog flow checks in the reservations are stale before making them as ineligible for dispatch. this is going to produce more dispatch pressure, so it might be wise to think of methods of progressive status update. maybe the engineer broker pushes a heartbeat every minute? 5 minutes? as edits on onto the reservation comment

Alternatives considered

just manually --force'ing them gets boring when your issue thoroughput is >50 a day

Before filing

I searched existing issues and this is not a duplicate.
This stays within ward's scope (the dev-verb gate / agent driver), not a personal-infra or downstream-repo verb.

### The problem when headless runs die, the director automated backlog flow just leaves them alone for forever b/c it thinks they are being worked on ### Proposed change backlog flow checks in the reservations are stale before making them as ineligible for dispatch. this is going to produce more dispatch pressure, so it might be wise to think of methods of progressive status update. maybe the engineer broker pushes a heartbeat every minute? 5 minutes? as edits on onto the reservation comment ### Alternatives considered just manually --force'ing them gets boring when your issue thoroughput is >50 a day ### Before filing - [x] I searched existing issues and this is not a duplicate. - [x] This stays within ward's scope (the dev-verb gate / agent driver), not a personal-infra or downstream-repo verb.

coilyco-ops added the

consult

labels

2026-07-10 09:00:28 +00:00

coilyco-ops commented

2026-07-10 09:03:07 +00:00

Member

Fresh evidence from the 2026-07-10 director sweep:

After redispatching ward#943, ward#690, and ward#905, the issues showed fresh reservation comments but did not appear in warded list shortly afterward, and no outcome/release comment was visible yet.

That creates exactly the failure mode this issue names: the backlog flow and a human director see a reserved issue and have to guess whether it is genuinely running, already exited, waiting for reap, or silently failed. warded logs coilyco-flight-deck/ward#943 --tail 120 was also ambiguous, printing only a source line and a container id with no explanatory log body.

This should feed the design: stale-reservation handling needs either a heartbeat/progress update on the reservation comment, or a director query path that can distinguish running, exited-no-outcome, waiting-reaper, and safe-to-redispatch without manual force/retry cycles.

Fresh evidence from the 2026-07-10 director sweep: After redispatching `ward#943`, `ward#690`, and `ward#905`, the issues showed fresh reservation comments but did not appear in `warded list` shortly afterward, and no outcome/release comment was visible yet. That creates exactly the failure mode this issue names: the backlog flow and a human director see a reserved issue and have to guess whether it is genuinely running, already exited, waiting for reap, or silently failed. `warded logs coilyco-flight-deck/ward#943 --tail 120` was also ambiguous, printing only a source line and a container id with no explanatory log body. This should feed the design: stale-reservation handling needs either a heartbeat/progress update on the reservation comment, or a director query path that can distinguish `running`, `exited-no-outcome`, `waiting-reaper`, and `safe-to-redispatch` without manual force/retry cycles.

coilyco-ops commented

2026-07-10 09:03:11 +00:00

Member

WARD-RESERVATION: held 🔒

reservation details

Holder: container engineer-codex-ward-978 on host kais-macbook-pro-2.local.

Reserved by ward agent --harness codex (reserved 2026-07-10T09:03:10Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (1h TTL). --force overrides.

Do not comment on or edit this issue to steer the run while it is reserved. The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a new issue, dispatched fresh. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).

run seed context — what this run is carrying (ward#609)

Resolved: coilyco-flight-deck/ward#978 · branch issue-978 · harness codex · workflow pull-requests-and-merge
Run: engineer-codex-ward-978 · ward v0.569.0 · dispatched 2026-07-10T09:03:10Z
Comment thread: 1 included in the pre-flight read, 0 stripped (ward's own automated comments).
- included: @coilyco-ops (2026-07-10T09:03:07Z)

Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.569.0).

— Codex, via ward agent

WARD-RESERVATION: held 🔒 <details><summary>reservation details</summary> Holder: container `engineer-codex-ward-978` on host `kais-macbook-pro-2.local`. Reserved by `ward agent --harness codex` (reserved 2026-07-10T09:03:10Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (1h TTL). `--force` overrides. **Do not comment on or edit this issue to steer the run while it is reserved.** The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a **new issue, dispatched fresh**. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494). <details><summary>run seed context — what this run is carrying (ward#609)</summary> - **Resolved:** `coilyco-flight-deck/ward#978` · branch `issue-978` · harness `codex` · workflow `pull-requests-and-merge` - **Run:** `engineer-codex-ward-978` · ward `v0.569.0` · dispatched `2026-07-10T09:03:10Z` - **Comment thread:** 1 included in the pre-flight read, 0 stripped (ward's own automated comments). - included: @coilyco-ops (2026-07-10T09:03:07Z) Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.569.0). </details> </details>  — Codex, via `ward agent`

coilyco-ops referenced this issue

2026-07-10 09:29:25 +00:00

Make Goose engineer runs terminate after final completion output #990

coilyco-ops referenced this issue from a pull request that will close it,

2026-07-10 09:30:09 +00:00

issues with stale reservations should be in scope for director backlog automated burndown flow #992

coilyco-ops commented

2026-07-10 09:31:08 +00:00

Member

WARD-OUTCOME: merge-ready

details

workflow: pull-requests-and-merge; review summary: skipped (review gate skipped by ~/.ward/config.yaml default)
retrospective: felt like a narrow reservation-state refactor. fresh holds now park, stale holds re-enter dispatch, and the reserved-run blind spot is covered.
confidence: high
surprise: the repo's test harness needed WARD_CONFIG_REF pinned for targeted director verification in this container, but the reservation slice passed cleanly.
follow-ups: watch PR #992 for merge, then let the director finish the landing.

WARD-OUTCOME: merge-ready <details><summary>details</summary> workflow: pull-requests-and-merge; review summary: skipped (review gate skipped by ~/.ward/config.yaml default) retrospective: felt like a narrow reservation-state refactor. fresh holds now park, stale holds re-enter dispatch, and the reserved-run blind spot is covered. confidence: high surprise: the repo's test harness needed `WARD_CONFIG_REF` pinned for targeted director verification in this container, but the reservation slice passed cleanly. follow-ups: watch PR #992 for merge, then let the director finish the landing. </details>

coilyco-ops closed this issue

2026-07-10 09:31:56 +00:00