coilyco-flight-deck/ward

Fork 0

Decide ward crash-reporting provider and scope before recovering salvage error reporting #668

New issue

Closed

opened 2026-07-08 08:45:30 +00:00 by coilyco-ops · 2 comments

coilyco-ops commented

2026-07-08 08:45:30 +00:00

Member

Goal

Decide whether ward should recover the error-reporting work from ward-salvage/ward-0d7db064, and if yes, what provider and scope are correct.

Context

The salvage audit in #666 classified ward-salvage/ward-0d7db064 as unclear, not safe to recover directly. The branch appears tied to ward#303 and adds real crash-reporting work:

cmd/ward/errorreport.go
cmd/ward/errorreport_test.go
docs/error-reporting.md
Sentry/GlitchTip dependency wiring in go.mod / go.sum
related docs and main.go changes

But ward#303 previously had a NO-GO note because the provider and scope were underspecified. The salvage branch picked Sentry/GlitchTip and a panic-only-ish boundary without that product decision being made.

Decision needed

Frame the options for Kai:

no crash reporting in ward for now, leave branch obsolete;
recover Sentry/GlitchTip crash reporting with a narrow panic/error boundary;
recover only local structured error artifacts, no external reporting;
another provider/scope.

Post a concise recommendation and consequences. Do not land code from this consult.

Filed from the read-only director surface (she/her).

## Goal Decide whether ward should recover the error-reporting work from `ward-salvage/ward-0d7db064`, and if yes, what provider and scope are correct. ## Context The salvage audit in #666 classified `ward-salvage/ward-0d7db064` as `unclear`, not safe to recover directly. The branch appears tied to ward#303 and adds real crash-reporting work: * `cmd/ward/errorreport.go` * `cmd/ward/errorreport_test.go` * `docs/error-reporting.md` * Sentry/GlitchTip dependency wiring in `go.mod` / `go.sum` * related docs and `main.go` changes But ward#303 previously had a NO-GO note because the provider and scope were underspecified. The salvage branch picked Sentry/GlitchTip and a panic-only-ish boundary without that product decision being made. ## Decision needed Frame the options for Kai: * no crash reporting in ward for now, leave branch obsolete; * recover Sentry/GlitchTip crash reporting with a narrow panic/error boundary; * recover only local structured error artifacts, no external reporting; * another provider/scope. Post a concise recommendation and consequences. Do not land code from this consult. Filed from the read-only director surface (she/her).

coilyco-ops added the

consult

labels

2026-07-08 08:45:30 +00:00

coilyco-ops commented

2026-07-08 08:47:54 +00:00

Author

Member

🔎 ward agent advisor

ward agent advisor --harness codex ran a one-shot standard research pass on this question:

Frame the decision for Kai: whether to recover ward-salvage/ward-0d7db064 error-reporting work, and if yes what provider and scope are correct. Read the salvage branch and ward#303 context. Post concise options, recommendation, consequences, and do not land code.

Recommendation

Recover the idea, not the salvage branch verbatim.

Recommended path: recover ward-salvage/ward-0d7db064 as fresh work from current main with this product decision locked first:

Provider: GlitchTip via github.com/getsentry/sentry-go, not Bugsnag and not hosted Sentry.
Phase-1 scope: host-side ward self-crash reporting only. Capture top-level ward Go panics, off by default, scrubbed through the existing redactSecrets path, DSN from env resolved by the operator layer from /sentry-dsn/ward.
Explicit non-goals for phase 1: no agent-outcome reporting, no transcript/content shipping to GlitchTip, no DSN injected into the warded container, no broad "all errors" boundary.

So Kai should not revive the branch as-is, but should treat it as a stale implementation of the right phase-1 direction.

Why

ward#303 was initially a real NO-GO. On 2026-06-25, the issue had only the title "ward => bugsnag / sentry integration". The pre-flight comment correctly stopped headless work because provider, scope, config source, and acceptance criteria were all unspecified.
The design question was later researched in the issue thread. On 2026-07-02, an advisor comment on ward#303 narrowed the intended design to GlitchTip via the Sentry SDK, reconciled against existing SigNoz telemetry, and proposed a phased rollout.
The salvage commit was created minutes later and matches that phase-1 design. Commit 0dae984 adds:
- cmd/ward/errorreport.go
- cmd/ward/errorreport_test.go
- docs/error-reporting.md
- main.go wiring for top-level panic capture
- getsentry/sentry-go
The boundary in the salvage commit is narrow and sane. It only captures ward panics, reuses redactSecrets, is off by default behind WARD_SENTRY_DSN, tags release/version/verb/host, and re-raises the panic after reporting.
The provider choice is already strongly implied by the surrounding repos. ward already has a GlitchTip operator surface, the fleet is in a Sentry -> GlitchTip cutover, /sentry-dsn/<slug> is the established DSN convention, and /substrate/agentic-os already ships a separate GlitchTip-backed tool-failure feed. Choosing Bugsnag or a different backend would cut across the current house direction.
SigNoz already owns the adjacent surface. cmd/ward/agent_telemetry.go and docs/agent-observability.md already send agent-run content and outcome to SigNoz, with a locality gate and redaction discipline. That means GlitchTip only adds value when it is scoped to "ward itself broke", not "the agent run failed".

Why not recover the branch directly

The salvage branch is not safe to cherry-pick blind.

It was salvaged from a run pinned at v0.319.0 on 2026-07-02.
Current main is now later than that, and the salvage branch diverges badly from current tree shape.
The commit itself is small, but its go.mod snapshot is stale enough to downgrade cli-guard from v0.75.0 to v0.65.0 in the salvaged diff. That is a strong sign the branch should be re-applied intentionally, not merged wholesale.

Options and consequences

1. Do nothing for now.
- Safest immediate choice.
- Consequence: ward's own panics remain visible only via stderr / logs / drain artifacts, with no grouped crash issues, regression signal, or alerting surface.
- Verdict: acceptable if Kai does not want to spend product attention here yet, but it leaves real value on the table.
2. Recover GlitchTip crash reporting with a narrow panic boundary.
- This is the recommended choice.
- Consequence: one new dependency, one new DSN/env seam, and a small panic-reporting path in cmd/ward, but the scope stays crisp and does not duplicate SigNoz.
- Recovery should start from current main, using the salvage commit as reference, not source of truth.
3. Recover only local structured error artifacts.
- Possible, but weaker.
- Consequence: this creates another local debugging surface while ward already has log drains, optional disk sink, and SigNoz for run telemetry. It still does not provide grouped issue tracking, first/last seen, regression detection, or alerting.
- Verdict: not the best trade unless Kai explicitly wants zero external error backend usage.
4. Pick another provider or broader scope.
- Not recommended.
- Consequence: Bugsnag / hosted Sentry would fight the already-established GlitchTip direction, and a broader "all errors / agent failures / container internals" scope would blur the SigNoz seam and increase secret-exposure risk.

Consequence of the recommendation

If Kai accepts the recommendation, the next engineering issue should say, in effect:

recover ward#303 from current main
use GlitchTip via sentry-go
keep phase 1 to host-side ward panic capture only
source DSN from operator-resolved env (WARD_SENTRY_DSN) backed by /sentry-dsn/ward
reuse the existing redaction discipline before send
do not inject a DSN into the warded container
do not report agent outcomes or transcript content to GlitchTip

Bottom line

Kai should not leave the branch obsolete, and should not merge the salvage branch directly.

Kai should recover the work as a fresh current-main implementation of the already-researched phase-1 design: GlitchTip-backed, off-by-default, host-side ward self-crash reporting with a narrow panic boundary.

Researched and posted automatically by ward agent advisor --harness codex (ward#179). This is one-shot research, not a carried change - verify before acting on it.

— Codex, via ward agent

### 🔎 ward agent advisor `ward agent advisor --harness codex` ran a one-shot **standard** research pass on this question: > Frame the decision for Kai: whether to recover ward-salvage/ward-0d7db064 error-reporting work, and if yes what provider and scope are correct. Read the salvage branch and ward#303 context. Post concise options, recommendation, consequences, and do not land code. --- ## Recommendation Recover the **idea**, not the salvage branch verbatim. **Recommended path:** recover `ward-salvage/ward-0d7db064` as **fresh work from current `main`** with this product decision locked first: - **Provider:** **GlitchTip via `github.com/getsentry/sentry-go`**, not Bugsnag and not hosted Sentry. - **Phase-1 scope:** **host-side ward self-crash reporting only**. Capture top-level `ward` Go panics, off by default, scrubbed through the existing `redactSecrets` path, DSN from env resolved by the operator layer from `/sentry-dsn/ward`. - **Explicit non-goals for phase 1:** no agent-outcome reporting, no transcript/content shipping to GlitchTip, no DSN injected into the warded container, no broad "all errors" boundary. So Kai should **not** revive the branch as-is, but should treat it as a stale implementation of the right **phase-1 direction**. ## Why - **`ward#303` was initially a real NO-GO.** On **2026-06-25**, the issue had only the title "ward => bugsnag / sentry integration". The pre-flight comment correctly stopped headless work because provider, scope, config source, and acceptance criteria were all unspecified. - **The design question was later researched in the issue thread.** On **2026-07-02**, an advisor comment on `ward#303` narrowed the intended design to **GlitchTip via the Sentry SDK**, reconciled against existing SigNoz telemetry, and proposed a phased rollout. - **The salvage commit was created minutes later and matches that phase-1 design.** Commit `0dae984` adds: - `cmd/ward/errorreport.go` - `cmd/ward/errorreport_test.go` - `docs/error-reporting.md` - `main.go` wiring for top-level panic capture - `getsentry/sentry-go` - **The boundary in the salvage commit is narrow and sane.** It only captures `ward` panics, reuses `redactSecrets`, is off by default behind `WARD_SENTRY_DSN`, tags release/version/verb/host, and re-raises the panic after reporting. - **The provider choice is already strongly implied by the surrounding repos.** `ward` already has a GlitchTip operator surface, the fleet is in a **Sentry -> GlitchTip** cutover, `/sentry-dsn/<slug>` is the established DSN convention, and `/substrate/agentic-os` already ships a separate **GlitchTip-backed tool-failure** feed. Choosing Bugsnag or a different backend would cut across the current house direction. - **SigNoz already owns the adjacent surface.** `cmd/ward/agent_telemetry.go` and `docs/agent-observability.md` already send **agent-run content and outcome** to SigNoz, with a locality gate and redaction discipline. That means GlitchTip only adds value when it is scoped to **"ward itself broke"**, not "the agent run failed". ## Why not recover the branch directly The salvage branch is **not** safe to cherry-pick blind. - It was salvaged from a run pinned at **`v0.319.0`** on **2026-07-02**. - Current `main` is now later than that, and the salvage branch diverges badly from current tree shape. - The commit itself is small, but its `go.mod` snapshot is stale enough to **downgrade `cli-guard` from `v0.75.0` to `v0.65.0`** in the salvaged diff. That is a strong sign the branch should be **re-applied intentionally**, not merged wholesale. ## Options and consequences - **1. Do nothing for now.** - Safest immediate choice. - Consequence: ward's own panics remain visible only via stderr / logs / drain artifacts, with no grouped crash issues, regression signal, or alerting surface. - Verdict: acceptable if Kai does not want to spend product attention here yet, but it leaves real value on the table. - **2. Recover GlitchTip crash reporting with a narrow panic boundary.** - This is the **recommended** choice. - Consequence: one new dependency, one new DSN/env seam, and a small panic-reporting path in `cmd/ward`, but the scope stays crisp and does not duplicate SigNoz. - Recovery should start from current `main`, using the salvage commit as reference, not source of truth. - **3. Recover only local structured error artifacts.** - Possible, but weaker. - Consequence: this creates another local debugging surface while `ward` already has log drains, optional disk sink, and SigNoz for run telemetry. It still does **not** provide grouped issue tracking, first/last seen, regression detection, or alerting. - Verdict: not the best trade unless Kai explicitly wants zero external error backend usage. - **4. Pick another provider or broader scope.** - Not recommended. - Consequence: Bugsnag / hosted Sentry would fight the already-established GlitchTip direction, and a broader "all errors / agent failures / container internals" scope would blur the SigNoz seam and increase secret-exposure risk. ## Consequence of the recommendation If Kai accepts the recommendation, the next engineering issue should say, in effect: - recover `ward#303` from current `main` - use **GlitchTip via `sentry-go`** - keep **phase 1** to **host-side `ward` panic capture only** - source DSN from operator-resolved env (`WARD_SENTRY_DSN`) backed by `/sentry-dsn/ward` - reuse the existing redaction discipline before send - do **not** inject a DSN into the warded container - do **not** report agent outcomes or transcript content to GlitchTip ## Bottom line Kai should **not leave the branch obsolete**, and should **not merge the salvage branch directly**. Kai should **recover the work as a fresh current-main implementation of the already-researched phase-1 design: GlitchTip-backed, off-by-default, host-side `ward` self-crash reporting with a narrow panic boundary.** --- Researched and posted automatically by `ward agent advisor --harness codex` (ward#179). This is one-shot research, not a carried change - verify before acting on it.   — Codex, via `ward agent`

coilyco-ops referenced this issue

2026-07-09 18:26:10 +00:00

Recover GlitchTip-backed host-side ward panic reporting #843

coilyco-ops commented

2026-07-09 18:26:21 +00:00

Author

Member

Consult interrogation: decision work is complete. The settled implementation is now #843, which carries the GlitchTip-backed, host-side panic-reporting recovery from current main. Closing this consult shell.

coilyco-ops closed this issue

2026-07-09 18:26:21 +00:00