coilyco-flight-deck/ward

Fork 0

Record partial launches when a running engineer lacks an issue reservation marker #1353

New issue

Closed

opened 2026-07-15 08:30:38 +00:00 by coilyco-ops · 3 comments

coilyco-ops commented

2026-07-15 08:30:38 +00:00

Member

Ops finding from the warded control plane coherence support lane on 2026-07-15.

While refreshing active Ward engineers, I found a live engineer container for #1346 even though the issue thread does not contain a reservation-held marker or any successful dispatch handoff.

Observed evidence:

ward agent list --json reports engineer-codex-ward-1346 as running for coilyco-flight-deck/ward, branch issue-1346.
docker ps --filter name=engineer-codex-ward-1346 shows the container was created at 2026-07-15 08:11:15 +0000 UTC and is still running.
docker inspect engineer-codex-ward-1346 shows labels ward.repo=coilyco-flight-deck/ward, ward.issue=1346, ward.workflow=pull-request-and-merge, and ward.driver=codex.
ward ops forgejo issue view coilyco-flight-deck ward 1346 --output json shows only two comments, both capacity-blocked notes. There is no  marker and no WARDED_WORKFLOW: reservation-held comment.
The latest #1346 comment at 2026-07-15T08:11:49Z says the dispatch retry was still blocked by repo capacity, after the container creation timestamp.

Why this matters:

Directors and the heartbeat depend on issue-thread reservation markers to avoid double-dispatching or treating work as idle.
A successful launch paired with capacity-blocked user-facing evidence is a false-negative dispatch outcome.
This can make active P0 work invisible from the issue surface even though a container is burning capacity.

Acceptance:

A launched engineer must leave a reservation marker on the issue before or atomically with the container becoming visible as running.
If issue comment creation fails after launch, Ward must surface that as an actionable partial-launch state in dispatch-health / agent list / user output, including repo, issue, container, and remediation.
A dispatch path must not both launch a container and report only repo-capacity failure to the director.
Add focused coverage for the success-after-capacity-check path and the comment-failure/partial-launch path.

Ops finding from the warded control plane coherence support lane on 2026-07-15. While refreshing active Ward engineers, I found a live engineer container for #1346 even though the issue thread does not contain a reservation-held marker or any successful dispatch handoff. Observed evidence: - `ward agent list --json` reports `engineer-codex-ward-1346` as `running` for `coilyco-flight-deck/ward`, branch `issue-1346`. - `docker ps --filter name=engineer-codex-ward-1346` shows the container was created at `2026-07-15 08:11:15 +0000 UTC` and is still running. - `docker inspect engineer-codex-ward-1346` shows labels `ward.repo=coilyco-flight-deck/ward`, `ward.issue=1346`, `ward.workflow=pull-request-and-merge`, and `ward.driver=codex`. - `ward ops forgejo issue view coilyco-flight-deck ward 1346 --output json` shows only two comments, both capacity-blocked notes. There is no `` marker and no `WARDED_WORKFLOW: reservation-held` comment. - The latest #1346 comment at `2026-07-15T08:11:49Z` says the dispatch retry was still blocked by repo capacity, after the container creation timestamp. Why this matters: - Directors and the heartbeat depend on issue-thread reservation markers to avoid double-dispatching or treating work as idle. - A successful launch paired with capacity-blocked user-facing evidence is a false-negative dispatch outcome. - This can make active P0 work invisible from the issue surface even though a container is burning capacity. Acceptance: - A launched engineer must leave a reservation marker on the issue before or atomically with the container becoming visible as running. - If issue comment creation fails after launch, Ward must surface that as an actionable partial-launch state in dispatch-health / agent list / user output, including repo, issue, container, and remediation. - A dispatch path must not both launch a container and report only repo-capacity failure to the director. - Add focused coverage for the success-after-capacity-check path and the comment-failure/partial-launch path. Related: - #1346 - #1347 - #1242

coilyco-ops added this to the warded control plane coherence milestone

2026-07-15 08:30:38 +00:00

coilyco-ops added the

coherence-core

headless

labels

2026-07-15 08:30:38 +00:00

coilyco-ops commented

2026-07-15 08:31:14 +00:00

Author

Member

Dispatch attempt from the QA/ops director at 2026-07-15T08:30Z was blocked by current Ward repo capacity.

Command attempted: warded engineer coilyco-flight-deck/ward#1353 --harness codex --workflow pull-request-and-merge.

Broker response:

repo engineer limit is reached for coilyco-flight-deck/ward: 3 active engineer(s) (limit 3); pass --override-capacity to exceed real running capacity
dispatch log: /Users/kai/.ward/agent-logs/dispatch/20260715T083046Z-director-codex-jp85-coilyco-flight-deck-ward-1353.log

The issue is captured and ready for dispatch when a Ward engineer slot clears.

Dispatch attempt from the QA/ops director at 2026-07-15T08:30Z was blocked by current Ward repo capacity. Command attempted: `warded engineer coilyco-flight-deck/ward#1353 --harness codex --workflow pull-request-and-merge`. Broker response: ```text repo engineer limit is reached for coilyco-flight-deck/ward: 3 active engineer(s) (limit 3); pass --override-capacity to exceed real running capacity dispatch log: /Users/kai/.ward/agent-logs/dispatch/20260715T083046Z-director-codex-jp85-coilyco-flight-deck-ward-1353.log ``` The issue is captured and ready for dispatch when a Ward engineer slot clears.

coilyco-ops referenced this issue

2026-07-15 08:33:31 +00:00

Brokered director dispatch does not honor --override-capacity when repo limit is full #1347

coilyco-ops commented

2026-07-15 08:37:08 +00:00

Author

Member

WARDED_WORKFLOW: reservation-held

reservation details

Holder: launch intent for container engineer-codex-ward-1353 on host kais-macbook-pro-2.local.

Accepted by ward agent --harness codex (reserved 2026-07-15T08:37:07Z). Concurrent ward agent runs are blocked until this intent becomes visible or the intent is released. The stale-intent fallback is still TTL-bounded (3h TTL). --override-reservation overrides.

Do not comment on or edit this issue to steer the run while it is reserved. The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a new issue, dispatched fresh. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).

run seed context — what this run is carrying (ward#609)

Resolved: coilyco-flight-deck/ward#1353 · branch issue-1353 · harness codex · workflow pull-request-and-merge
Run: engineer-codex-ward-1353 · ward v0.710.0 · dispatched 2026-07-15T08:37:03Z
Comment thread: 1 included in the pre-flight read, 0 stripped (ward's own automated comments).
- included: @coilyco-ops (2026-07-15T08:31:14Z)

Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.710.0).

— Codex, via ward agent

WARDED_WORKFLOW: reservation-held <details><summary>reservation details</summary> Holder: launch intent for container `engineer-codex-ward-1353` on host `kais-macbook-pro-2.local`. Accepted by `ward agent --harness codex` (reserved 2026-07-15T08:37:07Z). Concurrent `ward agent` runs are blocked until this intent becomes visible or the intent is released. The stale-intent fallback is still TTL-bounded (3h TTL). `--override-reservation` overrides. **Do not comment on or edit this issue to steer the run while it is reserved.** The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a **new issue, dispatched fresh**. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494). <details><summary>run seed context — what this run is carrying (ward#609)</summary> - **Resolved:** `coilyco-flight-deck/ward#1353` · branch `issue-1353` · harness `codex` · workflow `pull-request-and-merge` - **Run:** `engineer-codex-ward-1353` · ward `v0.710.0` · dispatched `2026-07-15T08:37:03Z` - **Comment thread:** 1 included in the pre-flight read, 0 stripped (ward's own automated comments). - included: @coilyco-ops (2026-07-15T08:31:14Z) Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.710.0). </details> </details>  — Codex, via `ward agent`

coilyco-ops referenced this issue from a pull request that will close it,

2026-07-15 08:48:29 +00:00

Record partial launches when a running engineer lacks an issue reservation marker #1359

coilyco-ops closed this issue

2026-07-15 08:49:57 +00:00

coilyco-ops referenced this issue

2026-07-15 08:50:59 +00:00

Finish dispatch-path partial-launch guarantees after #1359 #1360

coilyco-ops commented

2026-07-15 08:51:11 +00:00

Author

Member

QA follow-up after PR #1359 merged.

What passed:

PR #1359 merged at 2026-07-15T08:49:57Z with merge commit d3ef2bb2ae899d994facec5e8aea426ad8510877.
Forgejo test run 8387 passed.
Focused local check passed: go test ./cmd/ward -run 'PartialLaunch|DispatchHealth|AgentList|Reservation|RunningEngineer' -count=1.
The merged patch adds partial-launch visibility to ward agent list and ward agent dispatch-health.

What did not fully satisfy the issue:

The original acceptance also required the dispatch path not to launch a container while reporting only repo-capacity failure to the director.
PR #1359 did not touch the launch/broker dispatch path, and I did not find coverage for the success-after-capacity-check path.

I filed replacement issue #1360 with the same labels and milestone to finish the missing dispatch-path guarantee.

QA follow-up after PR #1359 merged. What passed: - PR #1359 merged at `2026-07-15T08:49:57Z` with merge commit `d3ef2bb2ae899d994facec5e8aea426ad8510877`. - Forgejo test run 8387 passed. - Focused local check passed: `go test ./cmd/ward -run 'PartialLaunch|DispatchHealth|AgentList|Reservation|RunningEngineer' -count=1`. - The merged patch adds `partial-launch` visibility to `ward agent list` and `ward agent dispatch-health`. What did not fully satisfy the issue: - The original acceptance also required the dispatch path not to launch a container while reporting only repo-capacity failure to the director. - PR #1359 did not touch the launch/broker dispatch path, and I did not find coverage for the success-after-capacity-check path. I filed replacement issue #1360 with the same labels and milestone to finish the missing dispatch-path guarantee.