Extract agent friction events during reap and drain

coilyco-ops commented

2026-07-14 07:26:40 +00:00

Member

Problem

A review of drained warded engineer logs showed that the useful friction signals are present at run teardown time but are currently left buried in console.log and transcript.jsonl.

Examples include missing transcripts, launch probe stalls, repeated tool failures, validation churn, malformed or missing WARD-OUTCOME comments, PR or branch state mismatches, reaper salvage, and unrelated residual extra-repo preservation. Once these are only raw logs, later review has to scrape ANSI-heavy terminal output and reconstruct context Ward already had during reap or drain.

Proposed change

Have Ward eagerly extract deterministic per-run friction events during reap and drain, next to the structured run summary from ward#1234.

This should be a cheap facts-only pass, not a fleet-level analyzer. Reap and drain should record evidence while the run context is still fresh:

target repo, issue, workflow, driver, and container
launch phase and transcript presence
raw and normalized outcome inputs
reaper state, preserved branch, salvage issue, and reason
PR, branch, main, and issue-comment signals already checked by the lifecycle
deterministic friction events with category, severity, confidence, fingerprint, and short evidence

Candidate artifact shape:

{
  "schema_version": 1,
  "run": "engineer-codex-ward-1140",
  "events": [
    {
      "stage": "reap",
      "category": "salvage-noise",
      "severity": "medium",
      "confidence": "high",
      "fingerprint": "reap/preserved-extra-repo/missing-closing-reference",
      "evidence": "ward container reap: preserved work on ..."
    }
  ]
}

The events may live in meta.json as friction[] or beside it as friction.jsonl. The deciding rule should be artifact ergonomics: meta.json stays the run index, while repeated event records may be easier to append as JSONL.

Non-goals

Do not add a ward agent logs analyze command in this issue.
Do not do weekly clustering, ranking, or LLM interpretation in reap.
Do not make teardown depend on expensive historical analysis.

Weekly or external review tooling can consume these artifacts later, but Ward should only emit the per-run facts it can know cheaply and reliably at drain time.

Acceptance criteria

Reap or drain emits structured friction events for headless engineer runs.
Missing transcript and prelaunch failure cases produce explicit events.
Reaper salvage and preserved branch cases produce explicit events with reasons.
Outcome mismatch cases are detectable without scraping console.log.
Existing log drains keep working if consumers ignore the new artifact or field.
Tests cover at least: clean landed run, green PR boundary, no-transcript launch failure, preserved salvage branch, and unrelated extra-repo residual preservation.

ward#1234 covers structured meta.json run summaries.
ward#1119 covers prompt evals. This issue supplies per-run friction evidence those evals or separate review tools can consume.

### Problem A review of drained warded engineer logs showed that the useful friction signals are present at run teardown time but are currently left buried in `console.log` and `transcript.jsonl`. Examples include missing transcripts, launch probe stalls, repeated tool failures, validation churn, malformed or missing `WARD-OUTCOME` comments, PR or branch state mismatches, reaper salvage, and unrelated residual extra-repo preservation. Once these are only raw logs, later review has to scrape ANSI-heavy terminal output and reconstruct context Ward already had during reap or drain. ### Proposed change Have Ward eagerly extract deterministic per-run friction events during reap and drain, next to the structured run summary from `ward#1234`. This should be a cheap facts-only pass, not a fleet-level analyzer. Reap and drain should record evidence while the run context is still fresh: - target repo, issue, workflow, driver, and container - launch phase and transcript presence - raw and normalized outcome inputs - reaper state, preserved branch, salvage issue, and reason - PR, branch, main, and issue-comment signals already checked by the lifecycle - deterministic friction events with category, severity, confidence, fingerprint, and short evidence Candidate artifact shape: ```json { "schema_version": 1, "run": "engineer-codex-ward-1140", "events": [ { "stage": "reap", "category": "salvage-noise", "severity": "medium", "confidence": "high", "fingerprint": "reap/preserved-extra-repo/missing-closing-reference", "evidence": "ward container reap: preserved work on ..." } ] } ``` The events may live in `meta.json` as `friction[]` or beside it as `friction.jsonl`. The deciding rule should be artifact ergonomics: `meta.json` stays the run index, while repeated event records may be easier to append as JSONL. ### Non-goals - Do not add a `ward agent logs analyze` command in this issue. - Do not do weekly clustering, ranking, or LLM interpretation in reap. - Do not make teardown depend on expensive historical analysis. Weekly or external review tooling can consume these artifacts later, but Ward should only emit the per-run facts it can know cheaply and reliably at drain time. ### Acceptance criteria - Reap or drain emits structured friction events for headless engineer runs. - Missing transcript and prelaunch failure cases produce explicit events. - Reaper salvage and preserved branch cases produce explicit events with reasons. - Outcome mismatch cases are detectable without scraping `console.log`. - Existing log drains keep working if consumers ignore the new artifact or field. - Tests cover at least: clean landed run, green PR boundary, no-transcript launch failure, preserved salvage branch, and unrelated extra-repo residual preservation. ### Related - `ward#1234` covers structured `meta.json` run summaries. - `ward#1119` covers prompt evals. This issue supplies per-run friction evidence those evals or separate review tools can consume.

coilyco-ops added the

headless

P2

labels

2026-07-14 07:28:06 +00:00

coilyco-ops commented

2026-07-14 09:01:31 +00:00

Author

Member

WARDED_WORKFLOW: reservation-held

reservation details

Holder: launch intent for container engineer-codex-ward-1237 on host kais-macbook-pro-2.local.

Accepted by ward agent --harness codex (reserved 2026-07-14T09:01:30Z). Concurrent ward agent runs are blocked until this intent becomes visible or the intent is released. The stale-intent fallback is still TTL-bounded (3h TTL). --override-reservation overrides.

Do not comment on or edit this issue to steer the run while it is reserved. The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a new issue, dispatched fresh. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).

run seed context — what this run is carrying (ward#609)

Resolved: coilyco-flight-deck/ward#1237 · branch issue-1237 · harness codex · workflow pull-request-and-merge
Run: engineer-codex-ward-1237 · ward v0.672.0 · dispatched 2026-07-14T09:01:26Z
Comment thread: 0 included in the pre-flight read, 0 stripped (ward's own automated comments).

Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.672.0).

— Codex, via ward agent

WARDED_WORKFLOW: reservation-held <details><summary>reservation details</summary> Holder: launch intent for container `engineer-codex-ward-1237` on host `kais-macbook-pro-2.local`. Accepted by `ward agent --harness codex` (reserved 2026-07-14T09:01:30Z). Concurrent `ward agent` runs are blocked until this intent becomes visible or the intent is released. The stale-intent fallback is still TTL-bounded (3h TTL). `--override-reservation` overrides. **Do not comment on or edit this issue to steer the run while it is reserved.** The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a **new issue, dispatched fresh**. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494). <details><summary>run seed context — what this run is carrying (ward#609)</summary> - **Resolved:** `coilyco-flight-deck/ward#1237` · branch `issue-1237` · harness `codex` · workflow `pull-request-and-merge` - **Run:** `engineer-codex-ward-1237` · ward `v0.672.0` · dispatched `2026-07-14T09:01:26Z` - **Comment thread:** 0 included in the pre-flight read, 0 stripped (ward's own automated comments). Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.672.0). </details> </details>  — Codex, via `ward agent`

coilyco-ops referenced this issue from coilyco-flight-deck/agentic-os

2026-07-14 10:09:11 +00:00

Add compact repo maps for high-churn warded workflow areas #524

coilysiren referenced this issue

2026-07-14 15:27:09 +00:00

director should emit colorized, space buffered, visually distinct messages about the categories of actions its performing on startup #1283

coilyco-ops added this to the warded control plane coherence milestone

2026-07-15 05:14:51 +00:00

coilyco-ops commented

2026-07-15 06:55:00 +00:00

Author

Member

Release triage note from Kai on 2026-07-15: headless carry is valid. Treat this as log parsing plus event taxonomy: extract structured friction events during reap and drain so directors can see why capacity was lost or runs stalled.

coilyco-ops commented

2026-07-15 07:16:35 +00:00

Author

Member

WARD-TRIAGE: warded control plane coherence milestone

This issue is part of the warded control plane coherence sprint. The release thesis is to make warded feel like one dependable control plane for agent work: higher safe parallelism, coherent config defaults, reliable broker/container behavior, human-feedback gates, and enough structured evidence for the next actor after a paused or failed run.

For this sprint, headless means an engineer should be able to carry the issue from current issue context to a merged change without new human decisions. If the issue discovers a missing decision, split or demote the unclear part instead of guessing.

WARD-TRIAGE: warded control plane coherence milestone This issue is part of the `warded control plane coherence` sprint. The release thesis is to make `warded` feel like one dependable control plane for agent work: higher safe parallelism, coherent config defaults, reliable broker/container behavior, human-feedback gates, and enough structured evidence for the next actor after a paused or failed run. For this sprint, `headless` means an engineer should be able to carry the issue from current issue context to a merged change without new human decisions. If the issue discovers a missing decision, split or demote the unclear part instead of guessing.

coilyco-ops commented

2026-07-15 07:53:25 +00:00

Author

Member

WARDED_WORKFLOW: reservation-held

reservation details

Holder: launch intent for container engineer-codex-ward-1237 on host kais-macbook-pro-2.local.

Accepted by ward agent --harness codex (reserved 2026-07-15T07:53:25Z). Concurrent ward agent runs are blocked until this intent becomes visible or the intent is released. The stale-intent fallback is still TTL-bounded (3h TTL). --override-reservation overrides.

Do not comment on or edit this issue to steer the run while it is reserved. The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a new issue, dispatched fresh. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).

run seed context — what this run is carrying (ward#609)

Resolved: coilyco-flight-deck/ward#1237 · branch issue-1237 · harness codex · workflow pull-request-and-merge
Run: engineer-codex-ward-1237 · ward v0.710.0 · dispatched 2026-07-15T07:53:19Z
Comment thread: 2 included in the pre-flight read, 1 stripped (ward's own automated comments).
- included: @coilyco-ops (2026-07-15T06:55:00Z), @coilyco-ops (2026-07-15T07:16:35Z)
- stripped: @coilyco-ops (2026-07-14T09:01:31Z)

Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.710.0).

— Codex, via ward agent

WARDED_WORKFLOW: reservation-held <details><summary>reservation details</summary> Holder: launch intent for container `engineer-codex-ward-1237` on host `kais-macbook-pro-2.local`. Accepted by `ward agent --harness codex` (reserved 2026-07-15T07:53:25Z). Concurrent `ward agent` runs are blocked until this intent becomes visible or the intent is released. The stale-intent fallback is still TTL-bounded (3h TTL). `--override-reservation` overrides. **Do not comment on or edit this issue to steer the run while it is reserved.** The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a **new issue, dispatched fresh**. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494). <details><summary>run seed context — what this run is carrying (ward#609)</summary> - **Resolved:** `coilyco-flight-deck/ward#1237` · branch `issue-1237` · harness `codex` · workflow `pull-request-and-merge` - **Run:** `engineer-codex-ward-1237` · ward `v0.710.0` · dispatched `2026-07-15T07:53:19Z` - **Comment thread:** 2 included in the pre-flight read, 1 stripped (ward's own automated comments). - included: @coilyco-ops (2026-07-15T06:55:00Z), @coilyco-ops (2026-07-15T07:16:35Z) - stripped: @coilyco-ops (2026-07-14T09:01:31Z) Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.710.0). </details> </details>  — Codex, via `ward agent`

coilyco-ops referenced this issue from a pull request that will close it,

2026-07-15 08:05:48 +00:00

Extract agent friction events during reap and drain #1350

coilyco-ops closed this issue

2026-07-15 08:07:56 +00:00

coilyco-ops commented

2026-07-15 08:08:34 +00:00

Author

Member

WARDED_WORKFLOW: #1350

details

workflow: pull-request-and-merge; review summary: review gate skipped because the temporary ward default pending brokered QA
felt: straightforward plumbing with one lint detour on the transcript-mode switch
confidence: high
surprises: Forgejo Actions was green on the first retry after the lint fix
follow-ups: none

WARDED_WORKFLOW: https://forgejo.coilysiren.me/coilyco-flight-deck/ward/pulls/1350 <details><summary>details</summary> workflow: pull-request-and-merge; review summary: review gate skipped because the temporary ward default pending brokered QA felt: straightforward plumbing with one lint detour on the transcript-mode switch confidence: high surprises: Forgejo Actions was green on the first retry after the lint fix follow-ups: none </details>

Rows
Columns

Extract agent friction events during reap and drain #1237

Problem

Proposed change

Non-goals

Acceptance criteria

Related