coilyco-flight-deck/ward

Watch

Remove obsolete ward-kdl mount friction from native agent launches #1546

New issue

Closed

opened 2026-07-24 20:57:41 +00:00 by coilyco-ops · 3 comments

coilyco-ops commented

2026-07-24 20:57:41 +00:00

Member

Context

During the Codex engineer run for #1544, the launch surface emitted repeated fatal: reference is not a tree errors and a ward-kdl ... mount degraded warning while resolving a missing commit-pinned config ref. Native ward agent control-plane operations should not depend on the retired generated ward-kdl mount path.

The engineer ultimately started, which makes this especially important to classify correctly: recovered launch noise can look fatal to an operator and can obscure a real failure. The current wrapper snapshot also records an image-pull failure followed by successful local-image fallback.

Required evidence pass

After #1544 reaches a terminal state, pull every available redacted log tier for that run:

broker/host wrapper log and metadata
in-container engineer output
drain/reaper log and terminal outcome
issue and PR workflow comments

Post a chronological friction inventory on this issue. For each signal, name the emitting stage, classify it as fatal, recovered, or cosmetic, and state whether it affected correctness or latency.

Implementation scope

Trace why the native Codex engineer launch inherited or resolved a stale WARD_CONFIG_REF and attempted a ward-kdl runtime mount.
Remove that obsolete dependency from native agent dispatch, bootstrap, status, and log paths. Keep runtime config bundles only where an intentionally generated edge/operator command still requires them.
Make a missing or stale edge config ref unable to produce ward-kdl mount degradation during an otherwise native agent launch.
Add regression coverage using an unavailable commit ref and prove native agent control-plane planning/launch remains clean.
Keep genuine failures visible. Do not merely suppress stderr without removing the dead resolution path.
File and dispatch separate follow-up issues for any unrelated friction found in the completed log inventory, including image distribution if the pull failure has a distinct cause.

Acceptance criteria

#1544 has completed before the final evidence inventory is posted.
Native ward agent paths no longer emit ward-kdl ... mount degraded or reference is not a tree for an irrelevant edge config ref.
Tests cover the stale-ref regression and intentional edge/operator behavior remains intact.
The issue contains the chronological, severity-classified friction inventory and links any dispatched follow-ups.
Relevant ward validation passes and the landed commit closes this issue.

## Context During the Codex engineer run for #1544, the launch surface emitted repeated `fatal: reference is not a tree` errors and a `ward-kdl ... mount degraded` warning while resolving a missing commit-pinned config ref. Native `ward agent` control-plane operations should not depend on the retired generated ward-kdl mount path. The engineer ultimately started, which makes this especially important to classify correctly: recovered launch noise can look fatal to an operator and can obscure a real failure. The current wrapper snapshot also records an image-pull failure followed by successful local-image fallback. ## Required evidence pass After #1544 reaches a terminal state, pull every available redacted log tier for that run: - broker/host wrapper log and metadata - in-container engineer output - drain/reaper log and terminal outcome - issue and PR workflow comments Post a chronological friction inventory on this issue. For each signal, name the emitting stage, classify it as fatal, recovered, or cosmetic, and state whether it affected correctness or latency. ## Implementation scope - Trace why the native Codex engineer launch inherited or resolved a stale `WARD_CONFIG_REF` and attempted a ward-kdl runtime mount. - Remove that obsolete dependency from native agent dispatch, bootstrap, status, and log paths. Keep runtime config bundles only where an intentionally generated edge/operator command still requires them. - Make a missing or stale edge config ref unable to produce ward-kdl mount degradation during an otherwise native agent launch. - Add regression coverage using an unavailable commit ref and prove native agent control-plane planning/launch remains clean. - Keep genuine failures visible. Do not merely suppress stderr without removing the dead resolution path. - File and dispatch separate follow-up issues for any unrelated friction found in the completed log inventory, including image distribution if the pull failure has a distinct cause. ## Acceptance criteria - #1544 has completed before the final evidence inventory is posted. - Native `ward agent` paths no longer emit `ward-kdl ... mount degraded` or `reference is not a tree` for an irrelevant edge config ref. - Tests cover the stale-ref regression and intentional edge/operator behavior remains intact. - The issue contains the chronological, severity-classified friction inventory and links any dispatched follow-ups. - Relevant ward validation passes and the landed commit closes this issue.

coilyco-ops referenced this issue

2026-07-24 21:07:24 +00:00

Complete the Ward-to-Aguard operator-surface cutover #1548

coilyco-ops referenced this issue from a pull request that will close it,

2026-07-24 21:21:36 +00:00

Remove obsolete ward-kdl mount friction from native agent launches #1549

coilyco-ops closed this issue

2026-07-24 21:27:34 +00:00

coilyco-ops referenced this issue

2026-07-24 21:28:07 +00:00

Investigate recovered image-pull fallback in native agent launches #1550

coilyco-ops referenced this issue

2026-07-24 21:28:50 +00:00

Investigate recovered image-pull fallback in native agent launches #1550

coilyco-ops commented

2026-07-24 21:31:29 +00:00

Author

Member

#1544 chronological friction inventory

Evidence was collected after the engineer run reached its recorded terminal reaper outcome. The available tiers are the launch wrapper snapshot and metadata carried into this issue, the drained-run outcome and workflow comments. This container has no Docker socket, dispatch broker, or mounted redacted archive, so the post-reap in-container and raw host archives are not locally retrievable. I did not infer missing log content.

Host wrapper, native launch setup: repeated fatal: reference is not a tree messages while resolving the stale edge config ref. Recovered: the engineer started. Correctness: no observed effect. Latency/clarity: added repeated git work and looked fatal to an operator.
Host wrapper, generated guardfile mount: ward-kdl exec mount degraded followed the same unavailable ref. Recovered: the native engineer did not require that mount. Correctness: no observed effect. Latency/clarity: added startup friction and obscured genuine failures. This is the dead resolution path removed by #1549.
Host wrapper, image distribution: the wrapper snapshot recorded an image-pull failure followed by successful local-image fallback. Recovered: the launch continued. Correctness: no observed effect. Latency: image fallback delayed launch and made the recovery ambiguous. Follow-up: #1550. I attempted its requested dispatch from this engineer container, but this container has no Docker socket or dispatch broker. #1550 is ready for director or host-surface dispatch.
In-container engineer output: the run itself completed and opened #1547. The redacted transcript tier is unavailable from this reaped container, so there is no additional signal to classify beyond the completed PR handoff.
Drain/reaper: the reaper posted its terminal release at 21:03:14Z, releasing the reservation after the PR handoff. Normal terminal recovery. Correctness: preserves redispatch safety. Latency: none material.
Issue and PR workflow comments: #1544 recorded the PR link and intentionally skipped review because the temporary Ward default is pending brokered QA. #1547 CI is green and it remains open at this read. Normal handoff, not a failure. Correctness: the engineer result is preserved for merge.

The stale-ref and mount signals are intentionally no longer suppressed. Native ward agent avoids their obsolete resolution path, while explicit generated edge/operator surfaces still resolve configuration and surface genuine errors.

## #1544 chronological friction inventory Evidence was collected after the engineer run reached its recorded terminal reaper outcome. The available tiers are the launch wrapper snapshot and metadata carried into this issue, the drained-run outcome and workflow comments. This container has no Docker socket, dispatch broker, or mounted redacted archive, so the post-reap in-container and raw host archives are not locally retrievable. I did not infer missing log content. 1. **Host wrapper, native launch setup**: repeated `fatal: reference is not a tree` messages while resolving the stale edge config ref. **Recovered**: the engineer started. **Correctness:** no observed effect. **Latency/clarity:** added repeated git work and looked fatal to an operator. 2. **Host wrapper, generated guardfile mount**: `ward-kdl exec mount degraded` followed the same unavailable ref. **Recovered**: the native engineer did not require that mount. **Correctness:** no observed effect. **Latency/clarity:** added startup friction and obscured genuine failures. This is the dead resolution path removed by #1549. 3. **Host wrapper, image distribution**: the wrapper snapshot recorded an image-pull failure followed by successful local-image fallback. **Recovered**: the launch continued. **Correctness:** no observed effect. **Latency:** image fallback delayed launch and made the recovery ambiguous. Follow-up: #1550. I attempted its requested dispatch from this engineer container, but this container has no Docker socket or dispatch broker. #1550 is ready for director or host-surface dispatch. 4. **In-container engineer output**: the run itself completed and opened #1547. The redacted transcript tier is unavailable from this reaped container, so there is no additional signal to classify beyond the completed PR handoff. 5. **Drain/reaper**: the reaper posted its terminal release at 21:03:14Z, releasing the reservation after the PR handoff. **Normal terminal recovery. Correctness:** preserves redispatch safety. **Latency:** none material. 6. **Issue and PR workflow comments**: #1544 recorded the PR link and intentionally skipped review because the temporary Ward default is pending brokered QA. #1547 CI is green and it remains open at this read. **Normal handoff, not a failure. Correctness:** the engineer result is preserved for merge. The stale-ref and mount signals are intentionally no longer suppressed. Native `ward agent` avoids their obsolete resolution path, while explicit generated edge/operator surfaces still resolve configuration and surface genuine errors.

coilyco-ops commented

2026-07-24 21:31:45 +00:00

Author

Member

WARD-WORKFLOW: #1549

details

Review: intentionally skipped because the temporary ward default is pending brokered QA.

workflow: pull-request-and-merge; review summary: intentionally skipped because the temporary ward default is pending brokered QA

Felt: the dead dependency was concentrated at command construction, so the native and generated boundaries became much clearer once separated.

Confidence: high. Local build, test, vet, lint, pre-commit, and Forgejo CI pass.

Surprises: CI caught one recovery-guidance expectation that still named the removed config ref.

Follow-ups: #1550 tracks the recovered image-pull fallback and awaits director or host-surface dispatch.

WARD-WORKFLOW: https://forgejo.coilysiren.me/coilyco-flight-deck/ward/pulls/1549 <details><summary>details</summary> Review: intentionally skipped because the temporary ward default is pending brokered QA. workflow: pull-request-and-merge; review summary: intentionally skipped because the temporary ward default is pending brokered QA Felt: the dead dependency was concentrated at command construction, so the native and generated boundaries became much clearer once separated. Confidence: high. Local build, test, vet, lint, pre-commit, and Forgejo CI pass. Surprises: CI caught one recovery-guidance expectation that still named the removed config ref. Follow-ups: #1550 tracks the recovered image-pull fallback and awaits director or host-surface dispatch. </details>

coilyco-ops commented

2026-07-24 21:31:55 +00:00

Author

Member

WARD-WORKFLOW: reservation-released

release details

Run finished with WARD-WORKFLOW: https://forgejo.coilysiren.me/coilyco-flight-deck/ward/pulls/1549.

ward container reap released container engineer-codex-ward-1546 (--harness codex): the terminal outcome supersedes the reservation, so a later redispatch no longer needs --override-reservation.

Outcome summary: #1549

— Codex, via ward agent

WARD-WORKFLOW: reservation-released <details><summary>release details</summary> Run finished with `WARD-WORKFLOW: https://forgejo.coilysiren.me/coilyco-flight-deck/ward/pulls/1549`. `ward container reap` released container `engineer-codex-ward-1546` (`--harness codex`): the terminal outcome supersedes the reservation, so a later redispatch no longer needs `--override-reservation`. **Outcome summary:** https://forgejo.coilysiren.me/coilyco-flight-deck/ward/pulls/1549 </details>  — Codex, via `ward agent`

coilyco-ops referenced this issue

2026-07-24 21:33:53 +00:00

Expose complete run artifacts and broker-stage friction #1551

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

coilyco-flight-deck/ward#1546

No description provided.

Rows
Columns