coilyco-flight-deck/otel-a2a-relay-debug
Archived
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Parser pipeline leaks timestamps + opaque ids into the public tier (normalize.text + ip_scrub insufficient) #3

New issue
Closed
opened 2026-06-06 06:11:22 +00:00 by coilysiren · 1 comment
coilysiren commented 2026-06-06 06:11:22 +00:00
Owner
Copy link

Summary

The normalize.text + ip_scrub parser pipeline leaks timestamps and opaque ids into the committed public/ tier for system-diagnostic probes. This violates the repo's own safety rule (AGENTS.md: "Never post a timestamp", "opaque values not"). It surfaced while running the on-box root-cause battery for the flapping-502 (o2r channel 9KP5, concept C1-onbox-rootcause) on kai-server.

What leaks

dbglib.anonymize() only runs ip_scrub (IPv4/IPv6 literals → <ip>) plus path-based redact. For normalize.text probes the payload is a flat {lines: [...], rc}, so path-redact can't reach inside lines. That leaves these un-scrubbed in public/current.json:

  • ISO timestamps — e.g. etcd lines time="2026-06-05T22:30:36-07:00" level=info msg="COMPACT ...". The T and single-colon offset dodge the IPv6 colon-run heuristic, so the full wall-clock timestamp survives.
  • Pod UIDs — 02ee3657-79f5-4611-975f-cb216259abee (opaque).
  • Container SHAs — f5f28b28c5cdf04... (opaque, ephemeral).

Probes affected by the same gap: k3s-journal, k3s-service (also emits the containerd-shim ... -id <sha> cgroup list and a since <timestamp> line), recent-reboots (boot/shutdown wall-clock times). host-pressure is clean (sizes/percentages only).

Incidental side effect worth noting: the IPv6 regex over-matches HH:MM:SS klog time-of-day tokens (22:29:06 → <ip>) because they look like hex colon-runs — so some timestamps get mangled-but-not-cleanly-removed, while the ISO ones pass through untouched. Either way the result is not the intended clean public tier.

Impact

  • Low live impact this round: every probe's first run bootstraps baseline = current, so the posted diff was (no change vs baseline) — nothing leaked to the channel. Outputs were kept local-only (not committed).
  • But the design intends public/ to be the committable, postable tier. As-is, any bless + re-run that produces a real diff on these probes, or any commit of their public/ tier, would publish timestamps + opaque ids.

Proposed fix

Add a normalize/anonymize stage for text probes (parser-driven, opt-in per id):

  • normalize.strip_timestamps: true — drop leading journald/klog timestamps and inline time="..." ISO stamps from each line (the diff substrate shouldn't carry wall-clock anyway; it's volatile).
  • anonymize.redact_patterns: [<regex>...] — regex redaction over lines for opaque tokens (UUIDs, 32/64-char hex SHAs), since path-redact can't reach inside a text line list.
  • Tighten the IPv6 heuristic so it stops eating HH:MM:SS.

Until this lands, k3s-journal / k3s-service / recent-reboots outputs should stay local-only and not be blessed-and-reposted.

Provenance

Found by the on-box worker (claude-linux-kai-server-up68) during C1 on channel 9KP5. Parsers + registry for the battery were authored in this repo (parsers/{k3s-service,k3s-journal,host-pressure,recent-reboots}.yml); they run correctly, but their public tier is not commit-safe until the above lands.

## Summary The `normalize.text` + `ip_scrub` parser pipeline leaks **timestamps and opaque ids** into the committed `public/` tier for system-diagnostic probes. This violates the repo's own safety rule (AGENTS.md: *"Never post a timestamp"*, *"opaque values not"*). It surfaced while running the on-box root-cause battery for the flapping-502 (o2r channel `9KP5`, concept `C1-onbox-rootcause`) on `kai-server`. ## What leaks `dbglib.anonymize()` only runs `ip_scrub` (IPv4/IPv6 literals → `<ip>`) plus path-based `redact`. For `normalize.text` probes the payload is a flat `{lines: [...], rc}`, so path-redact can't reach inside lines. That leaves these **un-scrubbed in `public/current.json`**: - **ISO timestamps** — e.g. etcd lines `time="2026-06-05T22:30:36-07:00" level=info msg="COMPACT ..."`. The `T` and single-colon offset dodge the IPv6 colon-run heuristic, so the full wall-clock timestamp survives. - **Pod UIDs** — `02ee3657-79f5-4611-975f-cb216259abee` (opaque). - **Container SHAs** — `f5f28b28c5cdf04...` (opaque, ephemeral). Probes affected by the same gap: `k3s-journal`, `k3s-service` (also emits the `containerd-shim ... -id <sha>` cgroup list and a `since <timestamp>` line), `recent-reboots` (boot/shutdown wall-clock times). `host-pressure` is clean (sizes/percentages only). Incidental side effect worth noting: the IPv6 regex *over*-matches `HH:MM:SS` klog time-of-day tokens (`22:29:06` → `<ip>`) because they look like hex colon-runs — so some timestamps get mangled-but-not-cleanly-removed, while the ISO ones pass through untouched. Either way the result is not the intended clean public tier. ## Impact - Low live impact this round: every probe's first run bootstraps `baseline = current`, so the posted diff was `(no change vs baseline)` — **nothing leaked to the channel**. Outputs were kept local-only (not committed). - But the design intends `public/` to be the committable, postable tier. As-is, any `bless` + re-run that produces a real diff on these probes, or any commit of their `public/` tier, would publish timestamps + opaque ids. ## Proposed fix Add a normalize/anonymize stage for `text` probes (parser-driven, opt-in per id): - `normalize.strip_timestamps: true` — drop leading journald/klog timestamps and inline `time="..."` ISO stamps from each line (the diff substrate shouldn't carry wall-clock anyway; it's volatile). - `anonymize.redact_patterns: [<regex>...]` — regex redaction over `lines` for opaque tokens (UUIDs, 32/64-char hex SHAs), since path-redact can't reach inside a text line list. - Tighten the IPv6 heuristic so it stops eating `HH:MM:SS`. Until this lands, `k3s-journal` / `k3s-service` / `recent-reboots` outputs should stay local-only and not be blessed-and-reposted. ## Provenance Found by the on-box worker (`claude-linux-kai-server-up68`) during C1 on channel `9KP5`. Parsers + registry for the battery were authored in this repo (`parsers/{k3s-service,k3s-journal,host-pressure,recent-reboots}.yml`); they run correctly, but their public tier is not commit-safe until the above lands.
coilysiren commented 2026-06-17 18:52:35 +00:00
Author
Owner
Copy link

Closing. o2r is archived in the June 2026 surface reduction - an optional agent channel, unused autonomously. Handover doctrine moves to human-mediated.

Closing. o2r is archived in the June 2026 surface reduction - an optional agent channel, unused autonomously. Handover doctrine moves to human-mediated.
Commenting is not possible because the repository is archived.
No Branch/Tag specified
Branches Tags
main
debug/land-recovery-battery
No results found.
Labels
Clear labels
  
coherence-core
Core review set for the warded control plane coherence milestone. These issues form the release spine; adjacent milestone issues are stretch or supporting work.

  
consult
A human decision, design, or external action must happen first. Fail-closed default for unlabeled/low-confidence.

  
headless
Agent can take it from open issue to merged change with no human in the loop. The auto-burndown queue.

  
interactive
Agent does the work but pauses at a human checkpoint mid-flight (design choice, destructive step, human-only verification).

  
P0
now / blocking / urgent

  
P1
next / important

  
P2
backlog / will do

  
P3
someday / low

  
P4
icebox / frozen / reopen if it becomes real

No labels
coherence-core
consult
headless
interactive
P0
P1
P2
P3
P4
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
coilyco-flight-deck/otel-a2a-relay-debug#3
No description provided.