coilyco-flight-deck/ward

Watch

Expand broker and host launch decision logging #1469

New issue

Closed

opened 2026-07-16 06:12:06 +00:00 by coilyco-ops · 2 comments

coilyco-ops commented

2026-07-16 06:12:06 +00:00

Member

Context from #1468:

The useful broker artifact was at ~/.ward/agent-logs/dispatch/<request>/console.log and redacted into /opt/ward-agent-logs/dispatch/<request>/console.redacted.log.
The log showed request id, requested argv, effective Ward version, resolved issue ref, launch plan, carrying line, full seeded prompt, reservation start/recheck/local acquire, image pull, env file write, drain waiter, and launch completed.
The seeded prompt block was many lines and mostly not useful for dispatch diagnosis.
The engineer then failed separately because /usr/local/bin/ward was attachment metadata JSON, not the binary. That failure is not this issue, but it exposed how sparse the launch decision logging is.

Desired behavior:

Emit one concise line for every meaningful broker/host launch decision checkpoint. Treat meaningful as non-trivial control flow that changes launch routing, filtering, queue/backpressure, reservation, version/source selection, asset/image choice, host/context setup, or container creation behavior.
Do not log every null/error check, but do log every branch whose outcome would matter to a human debugging launch admission or startup.
Keep repeated polling compact: log first/last and state changes, not identical loop iterations.
Move or collapse large payloads such as the seeded prompt, full issue body, full API responses, Docker inspect blobs, and registry chatter. The dispatch artifact can mention where the seed/payload lives instead of printing the whole thing inline.
Target roughly 50-200 lines for broker/host launch decisions, excluding the engineer harness transcript. The combined broker + host init + engineer init surface may be 100-400 lines, but Goose likely has the tightest stdout tolerance and should anchor the practical ceiling.

Acceptance criteria:

Brokered launch artifacts show the decision path clearly enough to answer why a run was accepted, deferred, refused, or handed to container startup.
Seeded prompt content no longer dominates the broker/host decision log.
Redacted and raw artifacts preserve the same decision breadcrumbs.
Tests cover at least the dispatch artifact shape for a representative engineer launch and one deferred/refused path.

Brokered engineer launch currently hides too much host-side decision logic. In coilyco-flight-deck/ward#1468 the dispatch artifact only made a handful of broker/launch decision points obvious before the large seeded prompt and then the reservation/container tail. Given how much control flow exists in broker admission, broker-time filters, host launch preflight, reservation, image/asset selection, and container bring-up, that is not enough to diagnose launch behavior. Context from #1468: - The useful broker artifact was at `~/.ward/agent-logs/dispatch/<request>/console.log` and redacted into `/opt/ward-agent-logs/dispatch/<request>/console.redacted.log`. - The log showed request id, requested argv, effective Ward version, resolved issue ref, launch plan, carrying line, full seeded prompt, reservation start/recheck/local acquire, image pull, env file write, drain waiter, and launch completed. - The seeded prompt block was many lines and mostly not useful for dispatch diagnosis. - The engineer then failed separately because `/usr/local/bin/ward` was attachment metadata JSON, not the binary. That failure is not this issue, but it exposed how sparse the launch decision logging is. Desired behavior: - Emit one concise line for every meaningful broker/host launch decision checkpoint. Treat meaningful as non-trivial control flow that changes launch routing, filtering, queue/backpressure, reservation, version/source selection, asset/image choice, host/context setup, or container creation behavior. - Do not log every null/error check, but do log every branch whose outcome would matter to a human debugging launch admission or startup. - Keep repeated polling compact: log first/last and state changes, not identical loop iterations. - Move or collapse large payloads such as the seeded prompt, full issue body, full API responses, Docker inspect blobs, and registry chatter. The dispatch artifact can mention where the seed/payload lives instead of printing the whole thing inline. - Target roughly 50-200 lines for broker/host launch decisions, excluding the engineer harness transcript. The combined broker + host init + engineer init surface may be 100-400 lines, but Goose likely has the tightest stdout tolerance and should anchor the practical ceiling. Acceptance criteria: - Brokered launch artifacts show the decision path clearly enough to answer why a run was accepted, deferred, refused, or handed to container startup. - Seeded prompt content no longer dominates the broker/host decision log. - Redacted and raw artifacts preserve the same decision breadcrumbs. - Tests cover at least the dispatch artifact shape for a representative engineer launch and one deferred/refused path.

coilyco-ops referenced this issue

2026-07-28 08:52:04 +00:00

Bound logs reads for fresh container-starting engineers with no transcript yet #1594

coilyco-ops added the

autonomy

headless

priority

labels

2026-07-28 08:55:14 +00:00

coilyco-ops referenced this issue from a commit

2026-07-28 10:00:36 +00:00

Expand broker launch decision logging

coilyco-ops closed this issue

2026-07-28 10:00:36 +00:00

coilyco-ops commented

2026-07-28 10:01:26 +00:00

Author

Member

WARD-WORKFLOW: done ✅

details

workflow: merge-remote-main; review summary: the in-container review gate was intentionally skipped because of the temporary ward default pending brokered QA.

Implementation felt direct after finding the broker artifact and host launch choke points; the main wrinkle was collapsing brokered seed output without changing direct launch logs.
Confidence: high.
Surprises: remote main advanced during push, so I merged it and reran validation.
Follow-ups: none.

WARD-WORKFLOW: done ✅ <details><summary>details</summary> workflow: merge-remote-main; review summary: the in-container review gate was intentionally skipped because of the temporary ward default pending brokered QA. Implementation felt direct after finding the broker artifact and host launch choke points; the main wrinkle was collapsing brokered seed output without changing direct launch logs. Confidence: high. Surprises: remote main advanced during push, so I merged it and reran validation. Follow-ups: none. </details>

coilyco-ops commented

2026-07-28 10:01:34 +00:00

Author

Member

WARD-WORKFLOW: reservation-released

release details

Run finished with WARD-WORKFLOW: done ✅.

ward container reap released container engineer-codex-ward-1469 (--harness codex): the terminal outcome supersedes the reservation, so a later redispatch no longer needs --override-reservation.

— Codex, via ward agent

WARD-WORKFLOW: reservation-released <details><summary>release details</summary> Run finished with `WARD-WORKFLOW: done ✅`. `ward container reap` released container `engineer-codex-ward-1469` (`--harness codex`): the terminal outcome supersedes the reservation, so a later redispatch no longer needs `--override-reservation`. </details>  — Codex, via `ward agent`

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

coilyco-flight-deck/ward#1469

No description provided.

Rows
Columns