coilyco-flight-deck/ward

Fork 0

Brokered director dispatch does not honor --override-capacity when repo limit is full #1347

New issue

Closed

opened 2026-07-15 07:59:28 +00:00 by coilyco-ops · 4 comments

coilyco-ops commented

2026-07-15 07:59:28 +00:00

Member

Ops finding from the warded control plane coherence support lane on 2026-07-15.

A read-only director surface needed to dispatch P0 replacement issue #1346 while the Ward repo was at its per-repo engineer limit. The normal dispatch failed as expected, but retries with --override-capacity also failed with the same hint saying to pass --override-capacity.

Commands attempted from the director surface:

warded engineer coilyco-flight-deck/ward#1346 --harness codex --workflow pull-request-and-merge --override-capacity
warded engineer --harness codex --workflow pull-request-and-merge --override-capacity coilyco-flight-deck/ward#1346

Observed result both times:

ward: dispatch broker: ward agent engineer --harness codex: repo engineer limit is reached for coilyco-flight-deck/ward: 3 active engineer(s) (limit 3); pass --override-capacity to exceed real running capacity

Why this matters:

The director surface advertises --override-capacity as the emergency path for exactly-one extra engineer.
Under milestone pressure, ops needs that path to work for P0 replacement or repair issues after stale work is stopped.
The error suggests the flag is either not propagated to the host broker, not accepted by the brokered path, or not applied to the repo-limit check.

Acceptance:

A brokered read-only director dispatch with --override-capacity reaches the host launch path and bypasses the intended capacity gate exactly as the non-broker path documents.
The dispatch log or user-facing output shows whether override-capacity was received and applied.
If policy intentionally forbids capacity override from brokered director surfaces, the CLI must fail with that explicit reason instead of telling the operator to pass a flag that cannot work.
Add focused coverage for brokered engineer dispatch argument propagation and repo-limit behavior.

#1346
#1045

Ops finding from the warded control plane coherence support lane on 2026-07-15. A read-only director surface needed to dispatch P0 replacement issue #1346 while the Ward repo was at its per-repo engineer limit. The normal dispatch failed as expected, but retries with `--override-capacity` also failed with the same hint saying to pass `--override-capacity`. Commands attempted from the director surface: ```bash warded engineer coilyco-flight-deck/ward#1346 --harness codex --workflow pull-request-and-merge --override-capacity warded engineer --harness codex --workflow pull-request-and-merge --override-capacity coilyco-flight-deck/ward#1346 ``` Observed result both times: ```text ward: dispatch broker: ward agent engineer --harness codex: repo engineer limit is reached for coilyco-flight-deck/ward: 3 active engineer(s) (limit 3); pass --override-capacity to exceed real running capacity ``` Why this matters: - The director surface advertises `--override-capacity` as the emergency path for exactly-one extra engineer. - Under milestone pressure, ops needs that path to work for P0 replacement or repair issues after stale work is stopped. - The error suggests the flag is either not propagated to the host broker, not accepted by the brokered path, or not applied to the repo-limit check. Acceptance: - A brokered read-only director dispatch with `--override-capacity` reaches the host launch path and bypasses the intended capacity gate exactly as the non-broker path documents. - The dispatch log or user-facing output shows whether override-capacity was received and applied. - If policy intentionally forbids capacity override from brokered director surfaces, the CLI must fail with that explicit reason instead of telling the operator to pass a flag that cannot work. - Add focused coverage for brokered engineer dispatch argument propagation and repo-limit behavior. Related: - #1346 - #1045

coilyco-ops added this to the warded control plane coherence milestone

2026-07-15 07:59:28 +00:00

coilyco-ops added the

headless

labels

2026-07-15 07:59:28 +00:00

coilyco-ops commented

2026-07-15 08:00:04 +00:00

Author

Member

Ops dispatch note from 2026-07-15.

I attempted to dispatch this issue with Codex immediately after filing it, but brokered dispatch failed with the same repo-limit path this issue describes:

warded engineer coilyco-flight-deck/ward#1347 --harness codex --workflow pull-request-and-merge

Result:

repo engineer limit is reached for coilyco-flight-deck/ward: 3 active engineer(s) (limit 3); pass --override-capacity to exceed real running capacity

This is left open in milestone 9 as an operational blocker. It should be dispatched as soon as Ward repo capacity clears, or by a surface where the intended override path actually reaches the host launch code.

Ops dispatch note from 2026-07-15. I attempted to dispatch this issue with Codex immediately after filing it, but brokered dispatch failed with the same repo-limit path this issue describes: ```bash warded engineer coilyco-flight-deck/ward#1347 --harness codex --workflow pull-request-and-merge ``` Result: ```text repo engineer limit is reached for coilyco-flight-deck/ward: 3 active engineer(s) (limit 3); pass --override-capacity to exceed real running capacity ``` This is left open in milestone 9 as an operational blocker. It should be dispatched as soon as Ward repo capacity clears, or by a surface where the intended override path actually reaches the host launch code.

coilyco-ops commented

2026-07-15 08:22:41 +00:00

Author

Member

WARDED_WORKFLOW: reservation-held

reservation details

Holder: launch intent for container engineer-codex-ward-1347 on host kais-macbook-pro-2.local.

Accepted by ward agent --harness codex (reserved 2026-07-15T08:22:41Z). Concurrent ward agent runs are blocked until this intent becomes visible or the intent is released. The stale-intent fallback is still TTL-bounded (3h TTL). --override-reservation overrides.

Do not comment on or edit this issue to steer the run while it is reserved. The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a new issue, dispatched fresh. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).

run seed context — what this run is carrying (ward#609)

Resolved: coilyco-flight-deck/ward#1347 · branch issue-1347 · harness codex · workflow pull-request-and-merge
Run: engineer-codex-ward-1347 · ward v0.710.0 · dispatched 2026-07-15T08:22:34Z
Comment thread: 1 included in the pre-flight read, 0 stripped (ward's own automated comments).
- included: @coilyco-ops (2026-07-15T08:00:04Z)

Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.710.0).

— Codex, via ward agent

WARDED_WORKFLOW: reservation-held <details><summary>reservation details</summary> Holder: launch intent for container `engineer-codex-ward-1347` on host `kais-macbook-pro-2.local`. Accepted by `ward agent --harness codex` (reserved 2026-07-15T08:22:41Z). Concurrent `ward agent` runs are blocked until this intent becomes visible or the intent is released. The stale-intent fallback is still TTL-bounded (3h TTL). `--override-reservation` overrides. **Do not comment on or edit this issue to steer the run while it is reserved.** The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a **new issue, dispatched fresh**. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494). <details><summary>run seed context — what this run is carrying (ward#609)</summary> - **Resolved:** `coilyco-flight-deck/ward#1347` · branch `issue-1347` · harness `codex` · workflow `pull-request-and-merge` - **Run:** `engineer-codex-ward-1347` · ward `v0.710.0` · dispatched `2026-07-15T08:22:34Z` - **Comment thread:** 1 included in the pre-flight read, 0 stripped (ward's own automated comments). - included: @coilyco-ops (2026-07-15T08:00:04Z) Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.710.0). </details> </details>  — Codex, via `ward agent`

coilyco-ops referenced this issue

2026-07-15 08:30:38 +00:00

Record partial launches when a running engineer lacks an issue reservation marker #1353

coilyco-ops referenced this issue from a pull request that will close it,

2026-07-15 08:31:13 +00:00

Honor brokered override-capacity at repo limit #1355

coilyco-ops closed this issue

2026-07-15 08:32:47 +00:00

coilyco-ops commented

2026-07-15 08:33:31 +00:00

Author

Member

QA verification after merged PR #1355 on 2026-07-15.

Checked the merged diff on current origin/main (ea5185045570ce215182112478120bc04e2978a2) against this issue acceptance:

brokered engineer argv now forwards --override-capacity in cmd/ward/agent_dispatch_broker_test.go coverage.
launchAgentContainer and maybeLaunchRepoEngineerBackpressure now pass c.Bool("override-capacity") to the repo-working backpressure gate rather than using reservation override.
launchRepoEngineerBackpressureCheck allows exactly one launch at the repo ceiling with an explicit warning and refuses when already past the ceiling.
focused coverage was added for both the at-limit override and over-limit refusal cases.

Focused verification run:

go test ./cmd/ward -run 'TestLaunchRepoEngineerBackpressure|TestAgentDispatchBrokerForwardsEngineerArgs' -count=1

Result: passed. This satisfies #1347; no replacement issue needed for this acceptance. Separate partial-launch/reservation evidence was captured in #1353.

QA verification after merged PR #1355 on 2026-07-15. Checked the merged diff on current `origin/main` (`ea5185045570ce215182112478120bc04e2978a2`) against this issue acceptance: - brokered engineer argv now forwards `--override-capacity` in `cmd/ward/agent_dispatch_broker_test.go` coverage. - `launchAgentContainer` and `maybeLaunchRepoEngineerBackpressure` now pass `c.Bool("override-capacity")` to the repo-working backpressure gate rather than using reservation override. - `launchRepoEngineerBackpressureCheck` allows exactly one launch at the repo ceiling with an explicit warning and refuses when already past the ceiling. - focused coverage was added for both the at-limit override and over-limit refusal cases. Focused verification run: ```bash go test ./cmd/ward -run 'TestLaunchRepoEngineerBackpressure|TestAgentDispatchBrokerForwardsEngineerArgs' -count=1 ``` Result: passed. This satisfies #1347; no replacement issue needed for this acceptance. Separate partial-launch/reservation evidence was captured in #1353.

coilyco-ops commented

2026-07-15 08:33:55 +00:00

Author

Member

WARDED_WORKFLOW: #1355

details

workflow: pull-request-and-merge; review summary: skipped (review gate intentionally skipped because the temporary ward default pending brokered QA)

The fix felt tight: one host-side gate change, one broker propagation assertion, and one focused capacity test path.
Confidence: high.
Surprises: the commit hook only wanted comment-length trimming, not code changes.
Follow-ups: none.

WARDED_WORKFLOW: https://forgejo.coilysiren.me/coilyco-flight-deck/ward/pulls/1355 <details><summary>details</summary> workflow: pull-request-and-merge; review summary: skipped (review gate intentionally skipped because the temporary ward default pending brokered QA) The fix felt tight: one host-side gate change, one broker propagation assertion, and one focused capacity test path. Confidence: high. Surprises: the commit hook only wanted comment-length trimming, not code changes. Follow-ups: none. </details>

coilyco-ops referenced this issue

2026-07-15 10:12:56 +00:00

Exclude closed issues from director dispatch proposals #1381