Store raw/normalized tiers in /tmp (out of the repo tree) so leak-prevention doesn't rely on .gitignore #7

New issue

Closed

opened 2026-06-07 02:32:18 +00:00 by coilysiren · 1 comment

coilysiren commented

2026-06-07 02:32:18 +00:00

Owner

Problem

The raw/ and normalized/ tiers hold un-anonymized capture (real IPs, FQDNs, the opaque tailnet suffix). Today they live inside the repo tree at outputs/<src>__<tgt>/<id>/{raw,normalized}/ and are kept out of git only by .gitignore. That makes .gitignore the single point of failure for a leak: a stray git add -f, an ignore-rule typo, a path rename, or a tool that bypasses ignore rules, and the real values land in committed history (and the GitHub mirror).

This already bit us: the ZHQK debug run left the opaque tailnet FQDN in the public tier and required a manual scrub before commit (see #3 and the land commit). The raw tier is the bigger blast radius if it ever escapes.

Proposal

Move the leak-bearing tiers out of the repo tree entirely - write raw/ (and normalized/) under a /tmp (or $TMPDIR) working dir instead of outputs/. Then:

They physically cannot be git added - they are not under the repo root, so no .gitignore dependency.
Only the public/ tier stays under outputs/ and is the sole thing git ever sees.
"Capture once, parse thrice" still holds: parsers are pure functions over the saved raw, so a /tmp raw path re-derives normalized + public the same way - just from outside the repo.

Sketch

dbglib grows a raw_root() / work_root() that resolves to e.g. ${TMPDIR:-/tmp}/o2r-debug/<src>__<tgt>/<id>/ for raw + normalized, while public/ stays at outputs/<src>__<tgt>/<id>/public/.
step.py / bless.py read+write raw and normalized from the work root; public unchanged.
Drop the raw/ + normalized/ entries from .gitignore once nothing writes them inside the tree (keep a guard so a stray in-tree raw still cannot commit).
README "Capture once, parse thrice" + the tier table updated to show raw/normalized living in the ephemeral work dir, public in-repo.

Tradeoff to note

/tmp is ephemeral - raw is lost on reboot / tmp cleanup, so a parser fix can no longer re-derive from an old capture after the box reboots. That is acceptable: re-deriving across a reboot is rare, and the leak-surface reduction is worth more. Call it out in the README.

Done when

Raw + normalized are written outside the repo tree; outputs/ holds only public/.
No reliance on .gitignore to keep un-anonymized capture out of git.
README reflects the new tier locations + the ephemerality tradeoff.

Relates to #3 (parser-level anonymization gaps) and #5 (outputs commit boundary).

## Problem The `raw/` and `normalized/` tiers hold un-anonymized capture (real IPs, FQDNs, the opaque tailnet suffix). Today they live inside the repo tree at `outputs/<src>__<tgt>/<id>/{raw,normalized}/` and are kept out of git **only** by `.gitignore`. That makes `.gitignore` the single point of failure for a leak: a stray `git add -f`, an ignore-rule typo, a path rename, or a tool that bypasses ignore rules, and the real values land in committed history (and the GitHub mirror). This already bit us: the ZHQK debug run left the opaque tailnet FQDN in the *public* tier and required a manual scrub before commit (see #3 and the land commit). The raw tier is the bigger blast radius if it ever escapes. ## Proposal Move the leak-bearing tiers **out of the repo tree entirely** - write `raw/` (and `normalized/`) under a `/tmp` (or `$TMPDIR`) working dir instead of `outputs/`. Then: - They physically cannot be `git add`ed - they are not under the repo root, so no `.gitignore` dependency. - Only the `public/` tier stays under `outputs/` and is the sole thing git ever sees. - "Capture once, parse thrice" still holds: parsers are pure functions over the saved raw, so a `/tmp` raw path re-derives normalized + public the same way - just from outside the repo. ## Sketch - `dbglib` grows a `raw_root()` / `work_root()` that resolves to e.g. `${TMPDIR:-/tmp}/o2r-debug/<src>__<tgt>/<id>/` for raw + normalized, while `public/` stays at `outputs/<src>__<tgt>/<id>/public/`. - `step.py` / `bless.py` read+write raw and normalized from the work root; public unchanged. - Drop the `raw/` + `normalized/` entries from `.gitignore` once nothing writes them inside the tree (keep a guard so a stray in-tree raw still cannot commit). - README "Capture once, parse thrice" + the tier table updated to show raw/normalized living in the ephemeral work dir, public in-repo. ## Tradeoff to note `/tmp` is ephemeral - raw is lost on reboot / tmp cleanup, so a parser fix can no longer re-derive from an old capture after the box reboots. That is acceptable: re-deriving across a reboot is rare, and the leak-surface reduction is worth more. Call it out in the README. ## Done when - Raw + normalized are written outside the repo tree; `outputs/` holds only `public/`. - No reliance on `.gitignore` to keep un-anonymized capture out of git. - README reflects the new tier locations + the ephemerality tradeoff. Relates to #3 (parser-level anonymization gaps) and #5 (outputs commit boundary).

coilysiren added the

label

2026-06-17 09:24:20 +00:00

coilysiren commented

2026-06-17 18:52:34 +00:00

Author

Owner

Closing. o2r is archived in the June 2026 surface reduction - an optional agent channel, unused autonomously. Handover doctrine moves to human-mediated.

coilysiren closed this issue

2026-06-17 18:52:35 +00:00

Commenting is not possible because the repository is archived.