Converge Ward friction shipping to ser8 SigNoz #637

New issue

Closed

opened 2026-07-24 21:34:46 +00:00 by coilyco-ops · 0 comments

coilyco-ops commented

2026-07-24 21:34:46 +00:00

Member

Problem

Ward writes structured per-run friction events to each native host under ~/.ward/agent-logs/<run>/meta.json, but no fleet service currently exports those facts. Full run-log SigNoz shipping was intentionally deferred and must remain off.

The narrow, secret-free OTLP producer is tracked in ward#1552.

Ownership

Ward owns the event and shipper contract. Infrastructure owns persistent host scheduling, endpoint configuration, rollout, and the SigNoz operational view.

Change

After ward#1552 ships, extend the native Ward tooling convergence for every host that launches or brokers Ward engineers.
Configure the friction shipper with the ser8 SigNoz OTLP/HTTP endpoint through declared role variables. Do not use a SigNoz API key or generated operator CLI.
Schedule the out-of-band shipper through the native service mechanism for each supported host class. Preserve its watermark/receipt state across reboots and package upgrades.
Keep the service least-privileged. It may read Ward meta.json files and its own state only, not console logs, transcripts, credentials, or arbitrary home-directory content.
Add a SigNoz dashboard or saved query for category, severity, fingerprint, repo, harness, host, and time. Add a bounded alert for high-severity repeated friction without paging on a single recovered event.
Document rollout, rollback, and a synthetic canary procedure that proves one event arrives with no body-shaped content.
Update docs/FEATURES.md.

Acceptance

Every native Ward launch host runs the shipper on a declared cadence after convergence.
Collector downtime leaves events pending and the next successful pass drains them once.
A synthetic canary is queryable in ser8 SigNoz by its fingerprint and host correlation fields.
The payload contains no console, transcript, prompt, tool body, credential, or opaque host identifier.
Disabling the role stops export without deleting local Ward archives or shipper receipts.
Existing full-log and transcript SigNoz shipping remains disabled.
Infrastructure validation and pre-commit pass.

Dependency

The rollout cannot complete until ward#1552 publishes the native shipper command.

## Problem Ward writes structured per-run friction events to each native host under `~/.ward/agent-logs/<run>/meta.json`, but no fleet service currently exports those facts. Full run-log SigNoz shipping was intentionally deferred and must remain off. The narrow, secret-free OTLP producer is tracked in [ward#1552](https://forgejo.coilysiren.me/coilyco-flight-deck/ward/issues/1552). ## Ownership Ward owns the event and shipper contract. Infrastructure owns persistent host scheduling, endpoint configuration, rollout, and the SigNoz operational view. ## Change * After ward#1552 ships, extend the native Ward tooling convergence for every host that launches or brokers Ward engineers. * Configure the friction shipper with the ser8 SigNoz OTLP/HTTP endpoint through declared role variables. Do not use a SigNoz API key or generated operator CLI. * Schedule the out-of-band shipper through the native service mechanism for each supported host class. Preserve its watermark/receipt state across reboots and package upgrades. * Keep the service least-privileged. It may read Ward `meta.json` files and its own state only, not console logs, transcripts, credentials, or arbitrary home-directory content. * Add a SigNoz dashboard or saved query for category, severity, fingerprint, repo, harness, host, and time. Add a bounded alert for high-severity repeated friction without paging on a single recovered event. * Document rollout, rollback, and a synthetic canary procedure that proves one event arrives with no body-shaped content. * Update `docs/FEATURES.md`. ## Acceptance * Every native Ward launch host runs the shipper on a declared cadence after convergence. * Collector downtime leaves events pending and the next successful pass drains them once. * A synthetic canary is queryable in ser8 SigNoz by its fingerprint and host correlation fields. * The payload contains no console, transcript, prompt, tool body, credential, or opaque host identifier. * Disabling the role stops export without deleting local Ward archives or shipper receipts. * Existing full-log and transcript SigNoz shipping remains disabled. * Infrastructure validation and pre-commit pass. ## Dependency The rollout cannot complete until ward#1552 publishes the native shipper command.

coilyco-ops added the

autonomy

live-collab

priority

labels

2026-07-29 03:46:31 +00:00

coilyco-ops closed this issue

2026-07-29 06:04:40 +00:00