Diagnose broker-forwarded dispatches that leave no reservation or running engineer #1418

New issue

Closed

opened 2026-07-15 12:00:13 +00:00 by coilyco-ops · 1 comment

coilyco-ops commented

2026-07-15 12:00:13 +00:00

Member

Problem

The director surface has now seen broker-forwarded engineer dispatches return success while no visible launch state appears afterward.

Current reproduction:

Immediate follow-up evidence:

ward agent list --json still shows only #1413 and #1335 running for coilyco-flight-deck/ward.
ward agent dispatch-health --repo coilyco-flight-deck/ward --json reports running=2, submitted=0, launch_intents=0, stale_prelaunch=0.
#1408 has no reservation comment from this successful forward; its latest comments are only prior DISPATCH-DEFERRED capacity notes.

This also matches an earlier suspicious #1408 attempt from this director session: the broker returned success with log /Users/kai/.ward/agent-logs/dispatch/20260715T114832Z-director-codex-jp85-coilyco-flight-deck-ward-1408.log, but no running container or reservation became visible afterward.

This is operationally important because a director can believe a P1 repair was dispatched while the lane has no durable issue-thread or ward agent list evidence that a sibling engineer exists.

Acceptance

Reproduce or explain the successful broker-forward/no-visible-launch path using the cited #1408 dispatch logs if available.
Make the brokered dispatch path leave one of these durable outcomes: a running/submitted engineer, a reservation/deferred issue comment, or a non-zero caller failure with the actionable reason.
Add regression coverage for the path where the host broker accepts/forwards a launch but the launch never becomes visible.
Leave final evidence showing a #1408-style retry either launches visibly or fails with a durable issue-thread/log trail.

Related: #1408, dispatch log /Users/kai/.ward/agent-logs/dispatch/20260715T115819Z-director-codex-jp85-coilyco-flight-deck-ward-1408.log.

## Problem The director surface has now seen broker-forwarded engineer dispatches return success while no visible launch state appears afterward. Current reproduction: Immediate follow-up evidence: - `ward agent list --json` still shows only #1413 and #1335 running for coilyco-flight-deck/ward. - `ward agent dispatch-health --repo coilyco-flight-deck/ward --json` reports running=2, submitted=0, launch_intents=0, stale_prelaunch=0. - #1408 has no reservation comment from this successful forward; its latest comments are only prior DISPATCH-DEFERRED capacity notes. This also matches an earlier suspicious #1408 attempt from this director session: the broker returned success with log /Users/kai/.ward/agent-logs/dispatch/20260715T114832Z-director-codex-jp85-coilyco-flight-deck-ward-1408.log, but no running container or reservation became visible afterward. This is operationally important because a director can believe a P1 repair was dispatched while the lane has no durable issue-thread or `ward agent list` evidence that a sibling engineer exists. ## Acceptance - Reproduce or explain the successful broker-forward/no-visible-launch path using the cited #1408 dispatch logs if available. - Make the brokered dispatch path leave one of these durable outcomes: a running/submitted engineer, a reservation/deferred issue comment, or a non-zero caller failure with the actionable reason. - Add regression coverage for the path where the host broker accepts/forwards a launch but the launch never becomes visible. - Leave final evidence showing a #1408-style retry either launches visibly or fails with a durable issue-thread/log trail. Related: #1408, dispatch log /Users/kai/.ward/agent-logs/dispatch/20260715T115819Z-director-codex-jp85-coilyco-flight-deck-ward-1408.log.

coilyco-ops added this to the warded control plane coherence milestone

2026-07-15 12:00:13 +00:00

coilyco-ops

2026-07-15 12:00:13 +00:00

closed this issue
added the
headless

P1
labels

coilyco-ops commented

2026-07-15 12:00:48 +00:00

Author

Member

WITHDRAWN: later refresh showed this was delayed visibility, not a confirmed lost dispatch. #1408 became visible as running container engineer-codex-ward-1408, and #1408 received a reservation comment at 2026-07-15T12:00:03Z for dispatch 2026-07-15T11:59:56Z. The issue body also lost part of the reproduction text due shell quoting while filing. Closing this instead of sending a sibling engineer after a false positive. If the same broker-forward/no-visible-launch state recurs after a longer wait with no reservation, file a fresh issue with body-file quoting and the final lane refresh evidence.