Scoped/redacted drain view for the director surface (narrow ward#525 to a redacted subset) #526

New issue

Closed

opened 2026-07-02 08:55:59 +00:00 by coilysiren · 2 comments

coilysiren commented

2026-07-02 08:55:59 +00:00

Owner

Follow-up to ward#525

ward#525 mounts the whole ~/.ward/agent-logs drain read-only into the director surface. That is the right first move, but the drained console.log and transcript.jsonl are raw - only the Slice-2 SigNoz envelope export is redacted (docs/agent-observability.md), not the on-disk artifacts. So a director reading the mount can see secrets that leaked to a past run's stdout (a Read of a .env, a token in a traceback) and other runs' full content. Accepted for now (host-local-unredacted already, same trust tier as ~/.ward/audit/), but the tighter version should exist.

Want

A redaction-at-rest path so the director surface reads a redacted view of the drain, not the raw stream. Sketch of options for the implementer to weigh:

Redacted sidecar artifact (preferred). At drain time, in addition to console.log/transcript.jsonl, write a console.redacted.log (and/or transcript.redacted.jsonl) by running the same redaction the envelope extractor already uses upstream of SigNoz (drop tool-result bodies, scrub known secret shapes). Point the ward#525 surface mount at only the redacted files + meta.json, leaving the raw ones unmounted. Directors get readable logs, raw secrets stay off the mount.
meta.json-only mount. Narrowest - mount just the secret-free allowlisted meta.jsons. Cheapest, but loses console/transcript visibility, which is most of the point, so this is the fallback, not the target.

Constraints

Reuse the existing extractor redaction, do not fork a second redactor (one source of truth for what is a secret).
Keep the raw console.log/transcript.jsonl on disk for the host-native drain + SigNoz path - this adds a redacted view, it does not replace the raw archive.
The surface mount (ward#525) then narrows to the redacted subset. Coordinate the mount-source change with whatever ward#525 landed.

Acceptance

The director surface sees a redacted console/transcript view with tool-result bodies dropped and known secret shapes scrubbed, plus meta.json.
The raw artifacts are not on the surface mount.
Redaction logic is shared with the envelope extractor, not duplicated.

Depends on

ward#525 (the read-only mount this narrows). Do this after #525 lands to avoid racing the same mount code.

## Follow-up to ward#525 ward#525 mounts the whole `~/.ward/agent-logs` drain read-only into the director surface. That is the right first move, but the drained `console.log` and `transcript.jsonl` are **raw** - only the Slice-2 SigNoz *envelope* export is redacted (`docs/agent-observability.md`), not the on-disk artifacts. So a director reading the mount can see secrets that leaked to a past run's stdout (a Read of a `.env`, a token in a traceback) and other runs' full content. Accepted for now (host-local-unredacted already, same trust tier as `~/.ward/audit/`), but the tighter version should exist. ## Want A redaction-at-rest path so the director surface reads a **redacted** view of the drain, not the raw stream. Sketch of options for the implementer to weigh: 1. **Redacted sidecar artifact (preferred).** At drain time, in addition to `console.log`/`transcript.jsonl`, write a `console.redacted.log` (and/or `transcript.redacted.jsonl`) by running the **same redaction the envelope extractor already uses** upstream of SigNoz (drop tool-result bodies, scrub known secret shapes). Point the ward#525 surface mount at only the redacted files + `meta.json`, leaving the raw ones unmounted. Directors get readable logs, raw secrets stay off the mount. 2. **meta.json-only mount.** Narrowest - mount just the secret-free allowlisted `meta.json`s. Cheapest, but loses console/transcript visibility, which is most of the point, so this is the fallback, not the target. ## Constraints - Reuse the existing extractor redaction, do not fork a second redactor (one source of truth for what is a secret). - Keep the raw `console.log`/`transcript.jsonl` on disk for the host-native drain + SigNoz path - this adds a redacted view, it does not replace the raw archive. - The surface mount (ward#525) then narrows to the redacted subset. Coordinate the mount-source change with whatever ward#525 landed. ## Acceptance - The director surface sees a redacted console/transcript view with tool-result bodies dropped and known secret shapes scrubbed, plus `meta.json`. - The raw artifacts are not on the surface mount. - Redaction logic is shared with the envelope extractor, not duplicated. ## Depends on - ward#525 (the read-only mount this narrows). Do this **after** #525 lands to avoid racing the same mount code.

coilyco-ops added the

consult

labels

2026-07-02 09:02:26 +00:00

coilyco-ops commented

2026-07-02 17:59:24 +00:00

Member

🔒 Reserved by ward agent --driver claude — container engineer-claude-ward-526 on host kais-macbook-pro-2.local is carrying this issue (reserved 2026-07-02T17:59:21Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); --force overrides.

— Claude (she/her), via ward agent

🔒 Reserved by `ward agent --driver claude` — container `engineer-claude-ward-526` on host `kais-macbook-pro-2.local` is carrying this issue (reserved 2026-07-02T17:59:21Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); `--force` overrides.  — Claude (she/her), via `ward agent`

coilysiren referenced this issue from a commit

2026-07-02 18:14:18 +00:00

feat(agent): redacted-at-rest drain view for the director surface (ward#526)

coilysiren closed this issue

2026-07-02 18:14:18 +00:00

coilysiren commented

2026-07-02 18:15:04 +00:00

Author

Owner

WARD-OUTCOME: done - drain now writes a redacted agent-logs-redacted/ sibling (shared extractor redaction) and the ward#525 surface mount binds it instead of the raw tree.

Retro: this one slotted in cleanly because ward#525 had already carved the seam — a mountOpts.AgentLogsDir opt-in threaded through buildUpPlan — so narrowing it was a one-line source swap plus a new parallel tree at drain time. The redaction reuse was the whole point and it was genuinely there for the taking: redactSecrets for the console and extractEnvelopes(_, true) for the transcript are exactly what the remote SigNoz export runs, so console.redacted.log / transcript.redacted.jsonl fell out without a second redactor.

Two things worth noting. First, a nice bonus fell out of using a separate tree rather than trying to filter within agent-logs/: the dispatch/ logs that also live under agent-logs/ were silently on the old whole-dir mount too, and they're off the redacted mount now for free. Second, the redacted view rides the disk-sink gate — under the signoz-only default nothing hits disk on either tree, so a director only sees logs when WARD_AGENT_SINK is disk/both. That matches the existing ward#525 behavior (the mount was already disk-sink-dependent), but it's a real usability caveat worth a follow-up if directors are expected to read logs under the default sink.

What fought back was entirely environmental, not the code: the shared host disk hit 100% mid-run (sibling containers), I made it worse by clearing the Go build cache, and the docs/agent-observability.md 80-line/4000-char cap forced several rounds of trimming since that doc was already 21 chars under the limit. Tests (redactConsole, redactedTranscript, writeRedactedArtifacts) passed green before and after the main merge. Confidence is high on correctness; the one soft spot is the disk-sink-gating UX noted above.

WARD-OUTCOME: done - drain now writes a redacted `agent-logs-redacted/` sibling (shared extractor redaction) and the ward#525 surface mount binds it instead of the raw tree. Retro: this one slotted in cleanly because ward#525 had already carved the seam — a `mountOpts.AgentLogsDir` opt-in threaded through `buildUpPlan` — so narrowing it was a one-line source swap plus a new parallel tree at drain time. The redaction reuse was the whole point and it was genuinely there for the taking: `redactSecrets` for the console and `extractEnvelopes(_, true)` for the transcript are exactly what the remote SigNoz export runs, so `console.redacted.log` / `transcript.redacted.jsonl` fell out without a second redactor. Two things worth noting. First, a nice bonus fell out of using a **separate** tree rather than trying to filter within `agent-logs/`: the `dispatch/` logs that also live under `agent-logs/` were silently on the old whole-dir mount too, and they're off the redacted mount now for free. Second, the redacted view rides the disk-sink gate — under the signoz-only default nothing hits disk on either tree, so a director only sees logs when `WARD_AGENT_SINK` is `disk`/`both`. That matches the existing ward#525 behavior (the mount was already disk-sink-dependent), but it's a real usability caveat worth a follow-up if directors are expected to read logs under the default sink. What fought back was entirely environmental, not the code: the shared host disk hit 100% mid-run (sibling containers), I made it worse by clearing the Go build cache, and the `docs/agent-observability.md` 80-line/4000-char cap forced several rounds of trimming since that doc was already 21 chars under the limit. Tests (`redactConsole`, `redactedTranscript`, `writeRedactedArtifacts`) passed green before and after the main merge. Confidence is high on correctness; the one soft spot is the disk-sink-gating UX noted above.