Make detached broker dispatch observable without a terminal

coilysiren commented

2026-07-29 16:39:16 +00:00

Owner

Outcome

Make every brokered dispatch fully inspectable after the originating terminal disconnects.

A goal-driven director or operator must be able to determine what was requested, whether it queued or launched, its current state, its last meaningful transition, and its terminal outcome using Ward-owned machine-readable surfaces and durable artifacts.

Public lifecycle

queued exists only for a persisted scheduling intent. Direct dispatch begins at accepted.
The public states are queued, accepted, launching, running, cleanup-needed, completed, blocked, failed, and interrupted.
blocked and interrupted are terminal for that request. A retry creates a new correlated request.
Orphan details are reason codes, not additional states.
Fine-grained internal journal phases remain diagnostic details rather than public lifecycle states.

Scope

Give every broker request and resulting run a stable correlation identity.
Expose the public lifecycle through retained queue, status, list, logs, and dispatch-artifact surfaces.
Record timestamps, target repository and issue, role, harness, workflow, last transition, and terminal reason without requiring the original terminal.
Preserve state across broker restart and make interrupted or orphaned requests distinguishable from active work.
Lead human output with the decisive current state and smallest next action.
Keep stored and rendered evidence secret-free and compatible with #1582.
Keep external collectors optional. Ward’s durable local state and machine-readable commands are the product contract.

Retention

Nonterminal and cleanup-needed records are never automatically pruned.
Terminal secret-free summaries remain until explicit pruning. The prune surface defaults to selecting records older than 30 days.
Prompt bodies, transcript bodies, and full logs have separate retention and are never required to determine lifecycle status.

Acceptance

A fully detached dispatch can be followed from request through terminal outcome using Ward commands alone.
Queue and status output distinguish waiting, active, interrupted, cleanup-needed, and terminal requests.
A broker restart preserves or reconstructs every nonterminal request without inventing duplicate work.
Human and JSON renderings agree on the authoritative state, outcome, and next action.
No credential, prompt body, transcript body, or unbounded command output is required for status correlation.
Tests cover success, refusal, launch failure, broker restart, orphan recovery, retention, pruning selection, and terminal cleanup.

Relationships

Coordinate redaction behavior with #1582.
Build on the stable broker request, run, sender, recipient, and message identities established by #1626. Do not create a parallel dispatch identity, transport, or lifecycle store.
Existing engineering recovery, logs, and stop behavior remain the compatibility baseline.
External telemetry, provider-specific exporters, and live deployment verification remain out of scope.

## Outcome Make every brokered dispatch fully inspectable after the originating terminal disconnects. A goal-driven director or operator must be able to determine what was requested, whether it queued or launched, its current state, its last meaningful transition, and its terminal outcome using Ward-owned machine-readable surfaces and durable artifacts. ## Public lifecycle * `queued` exists only for a persisted scheduling intent. Direct dispatch begins at `accepted`. * The public states are `queued`, `accepted`, `launching`, `running`, `cleanup-needed`, `completed`, `blocked`, `failed`, and `interrupted`. * `blocked` and `interrupted` are terminal for that request. A retry creates a new correlated request. * Orphan details are reason codes, not additional states. * Fine-grained internal journal phases remain diagnostic details rather than public lifecycle states. ## Scope * Give every broker request and resulting run a stable correlation identity. * Expose the public lifecycle through retained queue, status, list, logs, and dispatch-artifact surfaces. * Record timestamps, target repository and issue, role, harness, workflow, last transition, and terminal reason without requiring the original terminal. * Preserve state across broker restart and make interrupted or orphaned requests distinguishable from active work. * Lead human output with the decisive current state and smallest next action. * Keep stored and rendered evidence secret-free and compatible with #1582. * Keep external collectors optional. Ward’s durable local state and machine-readable commands are the product contract. ## Retention * Nonterminal and `cleanup-needed` records are never automatically pruned. * Terminal secret-free summaries remain until explicit pruning. The prune surface defaults to selecting records older than 30 days. * Prompt bodies, transcript bodies, and full logs have separate retention and are never required to determine lifecycle status. ## Acceptance * A fully detached dispatch can be followed from request through terminal outcome using Ward commands alone. * Queue and status output distinguish waiting, active, interrupted, cleanup-needed, and terminal requests. * A broker restart preserves or reconstructs every nonterminal request without inventing duplicate work. * Human and JSON renderings agree on the authoritative state, outcome, and next action. * No credential, prompt body, transcript body, or unbounded command output is required for status correlation. * Tests cover success, refusal, launch failure, broker restart, orphan recovery, retention, pruning selection, and terminal cleanup. ## Relationships * Coordinate redaction behavior with #1582. * Build on the stable broker request, run, sender, recipient, and message identities established by #1626. Do not create a parallel dispatch identity, transport, or lifecycle store. * Existing engineering recovery, logs, and stop behavior remain the compatibility baseline. * External telemetry, provider-specific exporters, and live deployment verification remain out of scope.

coilysiren added the

labels

2026-07-29 16:43:01 +00:00

coilyco-ops added

and removed

labels

2026-07-30 10:23:42 +00:00

coilyco-ops changed title from ~~Investigate / document / add o11y to broker dispatch queues running 100% headless - broker detached form a terminal~~ to Make detached broker dispatch observable without a terminal

2026-07-31 08:09:32 +00:00

coilyco-ops referenced this issue

2026-07-31 08:12:31 +00:00

Export secret-free run friction over OTLP #1552

coilyco-ops referenced this issue

2026-07-31 08:13:14 +00:00

Define discoverability contract for the post-ward-ops broker surface #1269

coilyco-ops referenced this issue

2026-07-31 08:13:54 +00:00

Broker vs pre-flight: check placement (duplicate where state can drift) + log directionality (broker->dispatcher, pre-flight->dispatchee, 2nd missing) #1077

coilyco-ops commented

2026-08-05 09:45:07 +00:00

Member

Decision resolved by Kai on 2026-08-05.

Active Ward scope: implement Ward-owned end-to-end lifecycle inspectability for detached broker requests and their resulting runs.

Public states:

queued
accepted
launching
running
cleanup-needed
completed
blocked
failed
interrupted

Semantics:

queued exists only for a persisted scheduling intent. Direct dispatch begins at accepted.
blocked and interrupted are terminal for that request. A retry creates a new correlated request.
Orphan details are reason codes, not additional states.
Fine-grained internal journal phases remain diagnostic details rather than public lifecycle states.

Retention:

Nonterminal and cleanup-needed records are never automatically pruned.
Terminal secret-free summaries remain until explicit pruning. The prune surface defaults to selecting records older than 30 days.
Prompt bodies, transcript bodies, and full logs have separate retention and are never required to determine lifecycle status.

External telemetry, provider-specific exporters, and live deployment verification remain out of scope. This decision clears the consult gate.

Decision resolved by Kai on 2026-08-05. Active Ward scope: implement Ward-owned end-to-end lifecycle inspectability for detached broker requests and their resulting runs. Public states: * queued * accepted * launching * running * cleanup-needed * completed * blocked * failed * interrupted Semantics: * queued exists only for a persisted scheduling intent. Direct dispatch begins at accepted. * blocked and interrupted are terminal for that request. A retry creates a new correlated request. * Orphan details are reason codes, not additional states. * Fine-grained internal journal phases remain diagnostic details rather than public lifecycle states. Retention: * Nonterminal and cleanup-needed records are never automatically pruned. * Terminal secret-free summaries remain until explicit pruning. The prune surface defaults to selecting records older than 30 days. * Prompt bodies, transcript bodies, and full logs have separate retention and are never required to determine lifecycle status. External telemetry, provider-specific exporters, and live deployment verification remain out of scope. This decision clears the consult gate.

coilyco-ops added

and removed

labels

2026-08-05 09:45:13 +00:00

coilyco-ops referenced this issue

2026-08-05 09:57:24 +00:00

Retire autonomous burndown in favor of harness-native goal orchestration #1620

coilyco-ops referenced this issue