dispatch broker: warded fails to resolve issue (forgejo get issue -> exit status 4), blocking sibling dispatch #596

New issue

Closed

opened 2026-07-04 05:10:10 +00:00 by coilysiren · 2 comments

coilysiren commented

2026-07-04 05:10:10 +00:00

Owner

Correction to the original framing

This issue was first filed as "grant the forgejo read tier (broker is write-only)." That premise was wrong and is retracted. The ward ops forgejo broker already serves issue read verbs from a director surface session: issue get, issue view, issue list, issue list-all, and issue search all return data. The only refused verb is ward ops forgejo describe (the meta verb that enumerates the operator surface), which comes back refused - out of tier. That single gate is minor and may well be intentional (it does not block reading any issue or repo state), so it is not the substance of this ticket. Filed the retraction here rather than closing, because a real bug surfaced in the same session.

The real bug: dispatch broker cannot resolve the issue

From a director surface session, dispatching a sibling engineer run fails deterministically (reproduced twice):

$ warded engineer coilyco-flight-deck/ward#596
ward agent: resolved issue ref coilyco-flight-deck/ward#596 -> coilyco-flight-deck/ward#596
ward: dispatch broker: ward agent engineer --driver claude: resolve issue coilyco-flight-deck/ward#596: forgejo: get issue coilyco-flight-deck/ward#596: exit status 4

The warded face resolves the ref fine, then hands to the host dispatch broker (WARD_DISPATCH_BROKER_ADDR), whose own forgejo: get issue step exits 4. This is a hard failure, not a policy refused, so it reads as a broken or mis-credentialed forgejo client on the host dispatch-broker side, distinct from the in-container ward ops forgejo read-broker (which reads issue #596 without trouble).

Impact

This blocks the whole point of a director surface session: it can file issues but cannot dispatch the sibling runs that carry them. Same class as ward#594 ("orphaned by a dispatch failure"). While it stands, every warded dispatch from a surface session dies at issue-resolve, so captured work strands on the board with no carrier.

What to check

The forgejo credential/base URL the host dispatch broker uses to get issue (exit 4 smells like an auth/404 from that client, not the in-container FORGEJO_TOKEN which reads fine).
Whether the broker should surface the underlying forgejo HTTP status instead of a bare exit status 4, so this is diagnosable without guessing.

Provenance

Surfaced scoping the node-stats-mcp auth overlay (coilyco-bridge/deploy#27) in a director surface session.

## Correction to the original framing This issue was first filed as "grant the forgejo read tier (broker is write-only)." **That premise was wrong** and is retracted. The `ward ops forgejo` broker already serves issue **read** verbs from a director surface session: `issue get`, `issue view`, `issue list`, `issue list-all`, and `issue search` all return data. The only refused verb is `ward ops forgejo describe` (the meta verb that enumerates the operator surface), which comes back `refused - out of tier`. That single gate is minor and may well be intentional (it does not block reading any issue or repo state), so it is not the substance of this ticket. Filed the retraction here rather than closing, because a real bug surfaced in the same session. ## The real bug: dispatch broker cannot resolve the issue From a director surface session, dispatching a sibling engineer run fails deterministically (reproduced twice): ``` $ warded engineer coilyco-flight-deck/ward#596 ward agent: resolved issue ref coilyco-flight-deck/ward#596 -> coilyco-flight-deck/ward#596 ward: dispatch broker: ward agent engineer --driver claude: resolve issue coilyco-flight-deck/ward#596: forgejo: get issue coilyco-flight-deck/ward#596: exit status 4 ``` The `warded` face resolves the ref fine, then hands to the host dispatch broker (`WARD_DISPATCH_BROKER_ADDR`), whose own `forgejo: get issue` step exits 4. This is a hard failure, not a policy `refused`, so it reads as a broken or mis-credentialed forgejo client on the **host dispatch-broker** side, distinct from the in-container `ward ops forgejo` read-broker (which reads issue #596 without trouble). ## Impact This blocks the whole point of a director surface session: it can **file** issues but cannot **dispatch** the sibling runs that carry them. Same class as ward#594 ("orphaned by a dispatch failure"). While it stands, every `warded` dispatch from a surface session dies at issue-resolve, so captured work strands on the board with no carrier. ## What to check - The forgejo credential/base URL the **host** dispatch broker uses to `get issue` (exit 4 smells like an auth/404 from that client, not the in-container FORGEJO_TOKEN which reads fine). - Whether the broker should surface the underlying forgejo HTTP status instead of a bare `exit status 4`, so this is diagnosable without guessing. ## Provenance Surfaced scoping the node-stats-mcp auth overlay (coilyco-bridge/deploy#27) in a director surface session.

coilysiren changed title from ~~dispatch broker: grant the forgejo read tier, not write-only~~ to dispatch broker: warded fails to resolve issue (forgejo get issue -> exit status 4), blocking sibling dispatch

2026-07-04 05:12:11 +00:00

coilyco-ops commented

2026-07-04 05:20:25 +00:00

Member

🔒 Reserved by ward agent --driver claude — container engineer-claude-ward-596 on host KAI-DESKTOP-TOWER is carrying this issue (reserved 2026-07-04T05:20:18Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); --force overrides.

Do not comment on or edit this issue to steer the run while it is reserved. The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a new issue, dispatched fresh — that is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).

The pre-flight judged this issue GO for an unattended run. Its justification:

pre-flight read (GO)

Pre-flight read only — judging from the issue text, not the local scratch directory.

Context to front-load: In the fresh clone I will read, before any edit: (1) docs/agent.md and docs/agent-preflight.md plus .agents/skills/tooling-ward-agent/SKILL.md for the dispatch flow and ref-resolution contract; (2) the ward-kdl trio (docs/ward-kdl.md, docs/ward-kdl-surface.md, docs/ward-kdl-in-ward.md) since the issue is flagged against that subsystem; (3) the dispatch-broker implementation itself — the code behind WARD_DISPATCH_BROKER_ADDR and its resolve issue / forgejo: get issue step — to see exactly how it constructs its forgejo client (env vs config file vs shelled-out CLI) and where "exit status 4" is swallowed; (4) the in-container ward ops forgejo read-broker path, as the working reference implementation to diff against; and (5) any broker credential/config wiring (how FORGEJO_TOKEN or its host-side equivalent reaches the broker process).

Main risk: the root cause may live partly in host state (the broker daemon's environment lacking or mis-scoping a forgejo credential), which an ephemeral container can neither reproduce nor restart — so end-to-end verification of the live dispatch path is likely impossible unattended. Mitigating that, the actionable substance is still in-repo: fix the credential/base-URL plumbing so the host broker resolves credentials the same way the working read-broker does, and surface the underlying forgejo HTTP status instead of a bare exit status 4, both testable with unit-level coverage. The residual unknown is whether the exit-4 turns out to be a 404 (wrong base URL / repo mapping) versus auth, which the code reading should disambiguate; if the fix genuinely requires a host-side redeploy or token rotation, the run should land the code fix plus diagnosability and say so plainly in the closing comment rather than claim live verification.

— Claude (she/her), via ward agent

🔒 Reserved by `ward agent --driver claude` — container `engineer-claude-ward-596` on host `KAI-DESKTOP-TOWER` is carrying this issue (reserved 2026-07-04T05:20:18Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); `--force` overrides. **Do not comment on or edit this issue to steer the run while it is reserved.** The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a **new issue, dispatched fresh** — that is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494). The pre-flight judged this issue **GO** for an unattended run. Its justification: <details><summary>pre-flight read (GO)</summary> Pre-flight read only — judging from the issue text, not the local scratch directory. **Context to front-load:** In the fresh clone I will read, before any edit: (1) `docs/agent.md` and `docs/agent-preflight.md` plus `.agents/skills/tooling-ward-agent/SKILL.md` for the dispatch flow and ref-resolution contract; (2) the ward-kdl trio (`docs/ward-kdl.md`, `docs/ward-kdl-surface.md`, `docs/ward-kdl-in-ward.md`) since the issue is flagged against that subsystem; (3) the dispatch-broker implementation itself — the code behind `WARD_DISPATCH_BROKER_ADDR` and its `resolve issue` / `forgejo: get issue` step — to see exactly how it constructs its forgejo client (env vs config file vs shelled-out CLI) and where "exit status 4" is swallowed; (4) the in-container `ward ops forgejo` read-broker path, as the working reference implementation to diff against; and (5) any broker credential/config wiring (how FORGEJO_TOKEN or its host-side equivalent reaches the broker process). Main risk: the root cause may live partly in **host state** (the broker daemon's environment lacking or mis-scoping a forgejo credential), which an ephemeral container can neither reproduce nor restart — so end-to-end verification of the live dispatch path is likely impossible unattended. Mitigating that, the actionable substance is still in-repo: fix the credential/base-URL plumbing so the host broker resolves credentials the same way the working read-broker does, and surface the underlying forgejo HTTP status instead of a bare `exit status 4`, both testable with unit-level coverage. The residual unknown is whether the exit-4 turns out to be a 404 (wrong base URL / repo mapping) versus auth, which the code reading should disambiguate; if the fix genuinely requires a host-side redeploy or token rotation, the run should land the code fix plus diagnosability and say so plainly in the closing comment rather than claim live verification. GO </details>  — Claude (she/her), via `ward agent`

coilysiren referenced this issue from a commit

2026-07-04 05:30:11 +00:00

fix(agent): fold ops-forgejo stderr into the error so dispatch failures name their cause

coilysiren closed this issue

2026-07-04 05:30:11 +00:00

coilysiren commented

2026-07-04 05:30:32 +00:00

Author

Owner

WARD-OUTCOME: done - forgejoClient.run now folds the ops-forgejo subprocess stderr into its error, so a dispatch get issue failure names its cause instead of a bare exit status 4.

The retraction in the body was the useful part of the ticket: it pointed me away from a tier-grant change and toward the real seam. Tracing exit status 4 through cli-guard's exitcode contract was the moment it clicked - 4 is Internal (the auth value-chain), not the HTTP 404 the original filing guessed (that would be 3). So the failure is almost certainly the host having neither FORGEJO_TOKEN nor SSM at broker time, and the actual bug I could fix deterministically is the opacity: the subprocess wrote the reason to stderr and Capture threw it into the run log, returning only the exit code.

The fix is a one-chokepoint change (run funnels every forgejo op), teed so live streaming and success-path warnings are untouched and only the error path gains detail. Confident in it - build/vet/lint/pre-commit green, and an end-to-end test drives a fake exit-4 exe and asserts the cause rides the error.

One rough edge worth a follow-up: I did not fix the underlying host-credential gap, because that's environmental (the host's FORGEJO_TOKEN/SSM state), not code. If director surfaces keep hitting this, a cheaper pre-check in forgejoTokenResolver that fails with an explicit 'no token source on this host' message (rather than leaning on the SSM/aws error) might be worth filing.

WARD-OUTCOME: done - forgejoClient.run now folds the ops-forgejo subprocess stderr into its error, so a dispatch `get issue` failure names its cause instead of a bare `exit status 4`. The retraction in the body was the useful part of the ticket: it pointed me away from a tier-grant change and toward the real seam. Tracing `exit status 4` through cli-guard's exitcode contract was the moment it clicked - 4 is `Internal` (the auth value-chain), not the HTTP 404 the original filing guessed (that would be 3). So the failure is almost certainly the host having neither `FORGEJO_TOKEN` nor SSM at broker time, and the actual bug I could fix deterministically is the *opacity*: the subprocess wrote the reason to stderr and `Capture` threw it into the run log, returning only the exit code. The fix is a one-chokepoint change (`run` funnels every forgejo op), teed so live streaming and success-path warnings are untouched and only the error path gains detail. Confident in it - build/vet/lint/pre-commit green, and an end-to-end test drives a fake exit-4 exe and asserts the cause rides the error. One rough edge worth a follow-up: I did *not* fix the underlying host-credential gap, because that's environmental (the host's FORGEJO_TOKEN/SSM state), not code. If director surfaces keep hitting this, a cheaper pre-check in `forgejoTokenResolver` that fails with an explicit 'no token source on this host' message (rather than leaning on the SSM/aws error) might be worth filing.