coilyco-flight-deck/ward

Fork 0

Per-role execution time limits (engineer 90m / advisor 60m / qa 30m / director none) + reservation TTL 3h > limit invariant #1074

New issue

Closed

opened 2026-07-10 18:47:32 +00:00 by coilyco-ops · 4 comments

coilyco-ops commented

2026-07-10 18:47:32 +00:00

Member

Per-role execution time limits + reservation-TTL invariant

Warded runs need a wall-clock execution time limit, per role, and the issue-reservation TTL must always exceed it. Today runs are unbounded (a hung goose run does not self-terminate - ward#990) and the reservation TTL (1h) can expire while a run is still going, which lets the slot be re-claimed mid-flight - the same double-dispatch failure mode as the ward#996 reservation race.

The limits (Kai, 2026-07-10)

role	execution time limit	notes
engineer	90 min	engineers observed hitting ~40 min regularly, so 90 gives headroom without letting a hung run burn hours
advisor	60 min
qa	30 min	shortest - it renders a verdict, it does not implement
director	none	interactive/attached, not a fire-and-forget run

(Rendered as a table for the spec; keep the shipped config as flat per-role values, not a prose table.)

Reservation-TTL invariant (the load-bearing rule)

Reservation TTL must always be strictly greater than the role's execution time limit. Otherwise a run outlives its own reservation, the hold goes stale while work is still in flight, and another dispatcher re-claims the slot - a double-dispatch (two engineers, one issue), exactly the ward#996 failure.

Default reservation TTL: 3 hours (raised from 1h), comfortably above the 90-min engineer limit.
Enforce the invariant in config: if any role's reservation TTL <= its execution limit, fail loud at load, never silently ship a config that can double-dispatch.

Do

Add per-role execution time limits to the ward-embedded role presets (engineer 90m / advisor 60m / qa 30m / director none), configurable.
On limit exceed: terminate the run and hand off to the normal reaper/salvage path (loose work committed or preserved on a salvage branch, reservation released). This also gives hung goose runs (ward#990) a hard stop.
Raise the default reservation TTL to 3h and add a load-time assertion that every role's reservation TTL > its execution limit.
Surface the countdown/limit in the run seed + warded list so a long run's remaining budget is visible, not a surprise kill.

Acceptance

An engineer run past 90m (advisor 60m, qa 30m) is terminated and reaped, not left hanging.
The director has no execution limit.
A config with reservation TTL <= any execution limit fails to load with a clear error.
With the 3h reservation vs 90m engineer limit, a run always completes-or-is-killed before its reservation can go stale, closing the stale-reservation double-dispatch path.

Context

Filed at Kai's direction. Lifecycle sibling of the ward#1069 anti-churn work: execution limits bound runaway/hung runs, and the reservation-TTL invariant closes the stale-hold double-dispatch race (ward#996). Limits live in the ward-embedded role presets; the reservation TTL + invariant live in ward's reservation system. Refs: ward#1069 (epic), ward#990 (goose non-termination), ward#996 (double-dispatch race), ward#1044 (duplicate-worker fix), .ward/roles.kdl (role-preset ownership note).

## Per-role execution time limits + reservation-TTL invariant Warded runs need a wall-clock execution time limit, per role, and the issue-reservation TTL must always exceed it. Today runs are unbounded (a hung goose run does not self-terminate - ward#990) and the reservation TTL (1h) can expire *while a run is still going*, which lets the slot be re-claimed mid-flight - the same double-dispatch failure mode as the ward#996 reservation race. ## The limits (Kai, 2026-07-10) | role | execution time limit | notes | | -------- | -------------------- | ----- | | engineer | **90 min** | engineers observed hitting ~40 min regularly, so 90 gives headroom without letting a hung run burn hours | | advisor | **60 min** | | | qa | **30 min** | shortest - it renders a verdict, it does not implement | | director | **none** | interactive/attached, not a fire-and-forget run | (Rendered as a table for the spec; keep the shipped config as flat per-role values, not a prose table.) ## Reservation-TTL invariant (the load-bearing rule) **Reservation TTL must always be strictly greater than the role's execution time limit.** Otherwise a run outlives its own reservation, the hold goes stale while work is still in flight, and another dispatcher re-claims the slot - a double-dispatch (two engineers, one issue), exactly the ward#996 failure. - Default reservation TTL: **3 hours** (raised from 1h), comfortably above the 90-min engineer limit. - Enforce the invariant in config: if any role's reservation TTL <= its execution limit, **fail loud at load**, never silently ship a config that can double-dispatch. ## Do - Add per-role execution time limits to the ward-embedded role presets (engineer 90m / advisor 60m / qa 30m / director none), configurable. - On limit exceed: terminate the run and hand off to the normal reaper/salvage path (loose work committed or preserved on a salvage branch, reservation released). This also gives hung goose runs (ward#990) a hard stop. - Raise the default reservation TTL to 3h and add a load-time assertion that every role's reservation TTL > its execution limit. - Surface the countdown/limit in the run seed + `warded list` so a long run's remaining budget is visible, not a surprise kill. ## Acceptance - An engineer run past 90m (advisor 60m, qa 30m) is terminated and reaped, not left hanging. - The director has no execution limit. - A config with reservation TTL <= any execution limit fails to load with a clear error. - With the 3h reservation vs 90m engineer limit, a run always completes-or-is-killed before its reservation can go stale, closing the stale-reservation double-dispatch path. ## Context Filed at Kai's direction. Lifecycle sibling of the ward#1069 anti-churn work: execution limits bound runaway/hung runs, and the reservation-TTL invariant closes the stale-hold double-dispatch race (ward#996). Limits live in the ward-embedded role presets; the reservation TTL + invariant live in ward's reservation system. Refs: ward#1069 (epic), ward#990 (goose non-termination), ward#996 (double-dispatch race), ward#1044 (duplicate-worker fix), `.ward/roles.kdl` (role-preset ownership note).

coilyco-ops added the

headless

labels

2026-07-10 18:47:32 +00:00

coilyco-ops commented

2026-07-10 22:58:27 +00:00

Author

Member

WARD-RESERVATION: held 🔒

reservation details

Holder: container engineer-codex-ward-1074 on host KAI-DESKTOP-TOWER.

Reserved by ward agent --harness codex (reserved 2026-07-10T22:58:27Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (1h TTL). --force overrides.

Do not comment on or edit this issue to steer the run while it is reserved. The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a new issue, dispatched fresh. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).

run seed context — what this run is carrying (ward#609)

Resolved: coilyco-flight-deck/ward#1074 · branch issue-1074 · harness codex · workflow pull-requests-and-merge
Run: engineer-codex-ward-1074 · ward v0.605.0 · dispatched 2026-07-10T22:58:27Z
Comment thread: 0 included in the pre-flight read, 0 stripped (ward's own automated comments).

Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.605.0).

— Codex, via ward agent

WARD-RESERVATION: held 🔒 <details><summary>reservation details</summary> Holder: container `engineer-codex-ward-1074` on host `KAI-DESKTOP-TOWER`. Reserved by `ward agent --harness codex` (reserved 2026-07-10T22:58:27Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (1h TTL). `--force` overrides. **Do not comment on or edit this issue to steer the run while it is reserved.** The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a **new issue, dispatched fresh**. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494). <details><summary>run seed context — what this run is carrying (ward#609)</summary> - **Resolved:** `coilyco-flight-deck/ward#1074` · branch `issue-1074` · harness `codex` · workflow `pull-requests-and-merge` - **Run:** `engineer-codex-ward-1074` · ward `v0.605.0` · dispatched `2026-07-10T22:58:27Z` - **Comment thread:** 0 included in the pre-flight read, 0 stripped (ward's own automated comments). Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.605.0). </details> </details>  — Codex, via `ward agent`

coilysiren referenced this issue

2026-07-10 23:05:40 +00:00

set director default timeout to 24 hours #1098

coilyco-ops referenced this issue from a pull request that will close it,

2026-07-10 23:27:56 +00:00

Per-role execution time limits (engineer 90m / advisor 60m / qa 30m / director none) + reservation TTL invariant #1103

coilyco-ops commented

2026-07-10 23:48:14 +00:00

Author

Member

WARD-OUTCOME: submitted

details

workflow: pull-request-and-merge; review summary: skip (in-container review gate intentionally skipped because the temporary ward default pending brokered QA)
implementation felt mechanical once the baked 3h default was restored.
confidence: high.
surprise: the no-config reservation path still read the 1h baked default.
follow-up: none.

WARD-OUTCOME: submitted <details><summary>details</summary> workflow: pull-request-and-merge; review summary: skip (in-container review gate intentionally skipped because the temporary ward default pending brokered QA) implementation felt mechanical once the baked 3h default was restored. confidence: high. surprise: the no-config reservation path still read the 1h baked default. follow-up: none. </details>

coilyco-ops commented

2026-07-11 00:24:28 +00:00

Author

Member

WARD-RESERVATION: held 🔒

reservation details

Holder: container engineer-codex-ward-1074 on host KAI-DESKTOP-TOWER.

Reserved by ward agent --harness codex (reserved 2026-07-11T00:24:28Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (1h TTL). --force overrides.

run seed context — what this run is carrying (ward#609)

Resolved: coilyco-flight-deck/ward#1074 · branch issue-1074 · harness codex · workflow pull-requests-and-merge
Run: engineer-codex-ward-1074 · ward v0.605.0 · dispatched 2026-07-11T00:24:28Z
Comment thread: 1 included in the pre-flight read, 1 stripped (ward's own automated comments).
- included: @coilyco-ops (2026-07-10T23:48:14Z)
- stripped: @coilyco-ops (2026-07-10T22:58:27Z)

Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.605.0).

— Codex, via ward agent

WARD-RESERVATION: held 🔒 <details><summary>reservation details</summary> Holder: container `engineer-codex-ward-1074` on host `KAI-DESKTOP-TOWER`. Reserved by `ward agent --harness codex` (reserved 2026-07-11T00:24:28Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (1h TTL). `--force` overrides. **Do not comment on or edit this issue to steer the run while it is reserved.** The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a **new issue, dispatched fresh**. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494). <details><summary>run seed context — what this run is carrying (ward#609)</summary> - **Resolved:** `coilyco-flight-deck/ward#1074` · branch `issue-1074` · harness `codex` · workflow `pull-requests-and-merge` - **Run:** `engineer-codex-ward-1074` · ward `v0.605.0` · dispatched `2026-07-11T00:24:28Z` - **Comment thread:** 1 included in the pre-flight read, 1 stripped (ward's own automated comments). - included: @coilyco-ops (2026-07-10T23:48:14Z) - stripped: @coilyco-ops (2026-07-10T22:58:27Z) Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.605.0). </details> </details>  — Codex, via `ward agent`

coilyco-ops commented

2026-07-11 00:39:50 +00:00

Author

Member

WARD-OUTCOME: submitted

details

workflow: pull-request-and-merge; review summary: skip (in-container review gate intentionally skipped because the temporary ward default pending brokered QA)
implementation felt mechanical once the role-budget/list merge was resolved.
confidence: high.
surprise: main had newer reservation-aware list phases, so the budget fields had to be reintroduced on top.
follow-up: none.
reservation: handed back.

WARD-OUTCOME: submitted <details><summary>details</summary> workflow: pull-request-and-merge; review summary: skip (in-container review gate intentionally skipped because the temporary ward default pending brokered QA) implementation felt mechanical once the role-budget/list merge was resolved. confidence: high. surprise: main had newer reservation-aware list phases, so the budget fields had to be reintroduced on top. follow-up: none. reservation: handed back. </details>

coilyco-ops closed this issue

2026-07-11 00:40:33 +00:00