Mount the issue-corpus into /substrate via coilysiren/inbox double-duty (consumer half of agentic-os#297) #575

Closed
opened 2026-07-03 17:50:30 +00:00 by coilysiren · 3 comments
Owner

What

The consumer half of the issue-corpus design (agentic-os#297). The producer landed - scripts/render-issue-corpus.py plus the hourly .forgejo/workflows/issue-corpus.yml cron render every tracked repo's full issue history (open AND closed, bodies + every comment) to a private git mirror. The design doc (agentic-os docs/issue-corpus.md) names the container-mount half as "a separate ward sibling issue, blocked on this one." That issue was never filed. This is it.

Design shift settled this session (Kai): inbox does double duty

Do not stand up a separate coilysiren/issue-corpus mirror repo. Make coilysiren/inbox do double duty: it is already private, already a corpus source (it is on scripts/issue-corpus-repos.txt), and it is the single reason the corpus must be private. inbox hosts the compiled corpus output too.

This collapses the design on both sides:

  • Producer (agentic-os, separate coordinated issue): repoint the renderer's MIRROR_REPO from coilysiren/issue-corpus to coilysiren/inbox (render into a subdir, e.g. corpus/). This deletes a whole rollout prerequisite - the doc's "provision the private mirror repo + its ISSUE_CORPUS_PUSH_TOKEN" step goes away, reusing inbox's existing private repo and push credential.
  • Consumer (this issue, ward): add coilysiren/inbox cache to cmd/ward/containerassets/preclone-repos.txt plus the denormalized ansible copy (infrastructure/scripts/clone-substrate-repos.sh). It reuses the existing cache-tier credentialed private-clone path (/etc/ward-git-credentials, written from the coilyco-ops FORGEJO_TOKEN, which already reads inbox), so warmSubstrateRepo (cmd/ward/container_bootstrap.go) drops it at /substrate/inbox. Genuinely ~2 lines of manifest.

Blocked on the producer actually pushing the corpus to inbox - do not merge the mount before the corpus lands there.

Constraints

  • Programmatic only, no LLM inference (Kai). The corpus inclusion is a deterministic render of issue bodies + comments. No model decides what is included. This matches the renderer as built - keep it that way.
  • Discovery index, not a source of truth. Grep it to locate an issue, then confirm live state via ward ops forgejo issue view <owner> <name> <N>. The rendered headers already carry that disclaimer - the mount must preserve the framing, not present the corpus as ground truth.
  • Never public. The corpus carries coilysiren/inbox, so it must stay a cache-tier entry - never baked into the shareable dev-base image seed (docker/dev-base/substrate-image-repos.txt / the image tier).

Open forks for the advisor design pass

  1. Layout - corpus into an inbox subdir (corpus/) so it does not collide with intake content, vs inbox's tree being effectively corpus-only. Depends on what inbox's git tree already holds (unreadable from the read-only director seat that filed this).
  2. Mount naming - the repo is inbox but the content is the cross-repo corpus, so /substrate/inbox reads oddly. Keep it, or override the mount dir name in warmSubstrateRepo.
  3. No-recursion confirmation - the renderer reads inbox's issues via the Forgejo API and writes git files, so inbox indexing itself is safe (different planes). Confirm the chosen subdir layout keeps that clean and the renderer never treats the corpus/ files as source content.
  4. Credential + trust surface - confirm the cache-tier clone of coilysiren/inbox authenticates via the existing coilyco-ops credential store, and that mounting a private corpus into every container matches the intended trust surface (all warded roles, or gated by role / context level).
  • agentic-os#297 - the producer this consumes, and agentic-os docs/issue-corpus.md.
  • cmd/ward/container_bootstrap.go warmSubstrateRepo / warmSubstrate, cmd/ward/containerassets/preclone-repos.txt (the cache tier).
  • ward#523 - sparse-mount substrate. Interacts: the corpus is rendered data, not a code repo with a trifecta reference surface, so the sparse-pattern likely does not apply to it - the advisor should note the interaction.
  • ward#573 - dependency auto-context. Different axis (declared deps vs the issue corpus), noted to avoid conflation.

Filed by the read-only director surface session (claude-linux-cbbb76a95881-cb37-she-her) capturing the unfiled consumer half of agentic-os#297, reframed around inbox double-duty.

## What The **consumer half** of the issue-corpus design ([agentic-os#297](https://forgejo.coilysiren.me/coilyco-flight-deck/agentic-os/issues/297)). The producer landed - `scripts/render-issue-corpus.py` plus the hourly `.forgejo/workflows/issue-corpus.yml` cron render every tracked repo's full issue history (open AND closed, bodies + every comment) to a private git mirror. The design doc (`agentic-os docs/issue-corpus.md`) names the container-mount half as *"a separate **ward** sibling issue, blocked on this one."* **That issue was never filed. This is it.** ## Design shift settled this session (Kai): inbox does double duty Do **not** stand up a separate `coilysiren/issue-corpus` mirror repo. Make **`coilysiren/inbox`** do double duty: it is already private, already a corpus **source** (it is on `scripts/issue-corpus-repos.txt`), and it is the single reason the corpus must be private. inbox hosts the compiled corpus **output** too. This collapses the design on both sides: - **Producer (agentic-os, separate coordinated issue):** repoint the renderer's `MIRROR_REPO` from `coilysiren/issue-corpus` to `coilysiren/inbox` (render into a subdir, e.g. `corpus/`). This **deletes a whole rollout prerequisite** - the doc's "provision the private mirror repo + its `ISSUE_CORPUS_PUSH_TOKEN`" step goes away, reusing inbox's existing private repo and push credential. - **Consumer (this issue, ward):** add `coilysiren/inbox cache` to [`cmd/ward/containerassets/preclone-repos.txt`](cmd/ward/containerassets/preclone-repos.txt) plus the denormalized ansible copy (`infrastructure/scripts/clone-substrate-repos.sh`). It reuses the existing **cache-tier credentialed private-clone path** (`/etc/ward-git-credentials`, written from the coilyco-ops `FORGEJO_TOKEN`, which already reads inbox), so `warmSubstrateRepo` (`cmd/ward/container_bootstrap.go`) drops it at `/substrate/inbox`. Genuinely ~2 lines of manifest. **Blocked on** the producer actually pushing the corpus to inbox - do not merge the mount before the corpus lands there. ## Constraints - **Programmatic only, no LLM inference (Kai).** The corpus inclusion is a deterministic render of issue bodies + comments. No model decides what is included. This matches the renderer as built - keep it that way. - **Discovery index, not a source of truth.** Grep it to locate an issue, then confirm live state via `ward ops forgejo issue view <owner> <name> <N>`. The rendered headers already carry that disclaimer - the mount must preserve the framing, not present the corpus as ground truth. - **Never public.** The corpus carries `coilysiren/inbox`, so it must stay a `cache`-tier entry - never baked into the shareable dev-base image seed (`docker/dev-base/substrate-image-repos.txt` / the `image` tier). ## Open forks for the advisor design pass 1. **Layout** - corpus into an inbox subdir (`corpus/`) so it does not collide with intake content, vs inbox's tree being effectively corpus-only. Depends on what inbox's git tree already holds (unreadable from the read-only director seat that filed this). 2. **Mount naming** - the repo is `inbox` but the content is the cross-repo corpus, so `/substrate/inbox` reads oddly. Keep it, or override the mount dir name in `warmSubstrateRepo`. 3. **No-recursion confirmation** - the renderer reads inbox's *issues* via the Forgejo API and writes *git files*, so inbox indexing itself is safe (different planes). Confirm the chosen subdir layout keeps that clean and the renderer never treats the `corpus/` files as source content. 4. **Credential + trust surface** - confirm the cache-tier clone of `coilysiren/inbox` authenticates via the existing coilyco-ops credential store, and that mounting a private corpus into every container matches the intended trust surface (all warded roles, or gated by role / context level). ## Related - [agentic-os#297](https://forgejo.coilysiren.me/coilyco-flight-deck/agentic-os/issues/297) - the producer this consumes, and `agentic-os docs/issue-corpus.md`. - `cmd/ward/container_bootstrap.go` `warmSubstrateRepo` / `warmSubstrate`, `cmd/ward/containerassets/preclone-repos.txt` (the `cache` tier). - [ward#523](https://forgejo.coilysiren.me/coilyco-flight-deck/ward/issues/523) - sparse-mount substrate. Interacts: the corpus is rendered data, not a code repo with a trifecta reference surface, so the sparse-pattern likely does not apply to it - the advisor should note the interaction. - [ward#573](https://forgejo.coilysiren.me/coilyco-flight-deck/ward/issues/573) - dependency auto-context. Different axis (declared deps vs the issue corpus), noted to avoid conflation. --- Filed by the read-only director surface session (claude-linux-cbbb76a95881-cb37-she-her) capturing the unfiled consumer half of agentic-os#297, reframed around inbox double-duty.
Author
Owner

Design proposal - inbox-double-duty issue-corpus mount

Design pass done in-session from the read-only director seat (the advisor path is 125'ing on host docker, #577 - so this substitutes for a warded advisor run rather than waiting on it). Grounded by a read-only clone of coilysiren/inbox and a read of the landed producer in /substrate/agentic-os.

Key finding that settles the layout: inbox's git tree is empty

A read-only clone of coilysiren/inbox shows one tracked file, a 4-byte README.md, and no corpus/. inbox is an issue-tracker-only repo - all its real content is intake issues, its git tree is a stub. So there is no tracked content to collide with, which resolves the whole "double duty" worry: the git plane is free for the corpus, the issue plane stays the intake tracker, and the two never touch (the renderer reads issues via the Forgejo API and writes files - different planes, no feedback loop).

Forks, resolved

  1. Layout -> corpus/ subdir. Even though the tree is empty, render under corpus/ (corpus/<owner>/<name>/<n>-<slug>.md + corpus/manifest.json), not the repo root. It labels the content, keeps README.md and any future intake files (templates, routing config) clear of the auto-committed index, and makes the mount's purpose obvious to a reader. Cost is zero because the renderer already writes to whatever --mirror-dir it is handed.
  2. Mount naming -> keep /substrate/inbox, no override machinery. warmSubstrateRepo mounts at /substrate/<name>, so this lands at /substrate/inbox with the index at /substrate/inbox/corpus/. Adding a mount-dir override just to rename it to /substrate/issue-corpus is new machinery for a cosmetic gain - not worth it. Document the /substrate/inbox/corpus/ path instead.
  3. No recursion -> confirmed safe. The renderer's source set is repos' issues (ward ops forgejo issue list-all), never git files, so it never re-ingests corpus/*.md. inbox's own issues render to corpus/coilysiren/inbox/*.md (expected - inbox is a source), issues->files, no loop. Pushing files to inbox's git tree does not touch issue updated_at, so the incremental manifest is stable across runs.
  4. Credential + trust -> reuse the existing cache tier, unconditional, no role gate. The container's system credential store (/etc/ward-git-credentials, coilyco-ops FORGEJO_TOKEN) already reads inbox - verified by cloning it this session. That is exactly how the existing private cache tier (coilyco-bridge/*) authenticates, so no new credential wiring. Mount it like any other cache-tier repo: unconditional for every run that warms substrate, honoring the existing WARD_SUBSTRATE_SKIP / context-level skips. Do not add a per-role gate - offline issue discovery is valuable to engineer, director, and advisor alike, and the corpus is already private so the trust surface matches what cache-tier repos already expose.

Constraints honored

  • Programmatic only, no LLM inference - the renderer is a deterministic markdown dump of issue bodies + comments. Nothing here adds a model in the path. Kept.
  • Discovery index, not a source of truth - the rendered headers already carry the "confirm live via ward ops forgejo issue view" disclaimer. The mount changes nothing about that. A ward-side docs note should repeat the grep-to-locate / API-to-confirm discipline.
  • Never public - stays a cache-tier entry, never baked into the shareable dev-base image seed.

The one real cost to flag

The corpus grows unbounded - every closed issue, forever, in every container's /substrate. Today that is small. Left unbounded it is a slowly-growing scan-and-disk surface on every run. Not a blocker now, worth a follow-up to bound it (a recency window, or skip the corpus at WARD_CONTEXT_LEVEL=0). This is distinct from #523's sparse-mount: the corpus is rendered data, not a code repo with a trifecta reference surface, so #523's trifecta-sparse pattern does not apply to it - a size bound would be its own mechanism.


Implementation checklist, split by repo

agentic-os (producer, coordinated - do first, the ward mount is blocked on the corpus actually landing in inbox):

  • .forgejo/workflows/issue-corpus.yml: set MIRROR_REPO: coilysiren/inbox.
  • Same workflow: after cloning the mirror, mkdir -p "${MIRROR_DIR}/corpus" and call render-issue-corpus.py --mirror-dir "${MIRROR_DIR}/corpus". No renderer code change - the layout is --mirror-dir-controlled. git add -A from the clone root then commits corpus/ and leaves README.md untouched.
  • Confirm the trufflehog scan still covers the rendered tree (the renderer scans its --mirror-dir, now corpus/) before the push.
  • Point the push credential (ISSUE_CORPUS_PUSH_TOKEN, or reuse the coilyco-ops write token) at write access to coilysiren/inbox. Retire the separate coilysiren/issue-corpus mirror repo and its dedicated provisioning - that rollout prerequisite is now gone.
  • Update docs/issue-corpus.md: MIRROR_REPO is inbox, output under corpus/, drop the "provision a new private mirror repo" bullet.

ward (consumer - this issue, after the corpus is live in inbox):

  • Add coilysiren/inbox cache to cmd/ward/containerassets/preclone-repos.txt (cache tier - private, never image-baked).
  • Add the same line to the denormalized ansible copy, infrastructure/scripts/clone-substrate-repos.sh (the two stay denormalized per that file's own note).
  • No warmSubstrateRepo code change needed - the cache tier already does the credentialed private clone. Mount appears at /substrate/inbox, index at /substrate/inbox/corpus/.
  • Docs note (ward substrate docs) pointing agents at /substrate/inbox/corpus/ as the greppable discovery index, repeating the confirm-live-via-ward ops forgejo issue view discipline.
  • docs/FEATURES.md entry if this counts as a user/agent-visible capability (offline issue-corpus grep).

follow-up (separate issue, do not fold in):

  • Bound the corpus growth (recency window, or skip at WARD_CONTEXT_LEVEL=0) once its size warrants it.

Design pass by the read-only director surface session (claude-linux-cbbb76a95881-cb37-she-her), in-session substitute for the advisor run blocked by #577. Ready for an engineer dispatch once the agentic-os producer change lands the corpus in inbox.

## Design proposal - inbox-double-duty issue-corpus mount Design pass done **in-session from the read-only director seat** (the advisor path is 125'ing on host docker, [#577](https://forgejo.coilysiren.me/coilyco-flight-deck/ward/issues/577) - so this substitutes for a `warded advisor` run rather than waiting on it). Grounded by a read-only clone of `coilysiren/inbox` and a read of the landed producer in `/substrate/agentic-os`. ### Key finding that settles the layout: inbox's git tree is empty A read-only clone of `coilysiren/inbox` shows **one tracked file, a 4-byte `README.md`, and no `corpus/`**. inbox is an **issue-tracker-only** repo - all its real content is intake *issues*, its git tree is a stub. So there is **no tracked content to collide with**, which resolves the whole "double duty" worry: the git plane is free for the corpus, the issue plane stays the intake tracker, and the two never touch (the renderer reads *issues* via the Forgejo API and writes *files* - different planes, no feedback loop). ### Forks, resolved 1. **Layout -> `corpus/` subdir.** Even though the tree is empty, render under `corpus/` (`corpus/<owner>/<name>/<n>-<slug>.md` + `corpus/manifest.json`), not the repo root. It labels the content, keeps `README.md` and any future intake files (templates, routing config) clear of the auto-committed index, and makes the mount's purpose obvious to a reader. Cost is zero because the renderer already writes to whatever `--mirror-dir` it is handed. 2. **Mount naming -> keep `/substrate/inbox`, no override machinery.** `warmSubstrateRepo` mounts at `/substrate/<name>`, so this lands at `/substrate/inbox` with the index at `/substrate/inbox/corpus/`. Adding a mount-dir override just to rename it to `/substrate/issue-corpus` is new machinery for a cosmetic gain - not worth it. Document the `/substrate/inbox/corpus/` path instead. 3. **No recursion -> confirmed safe.** The renderer's source set is repos' *issues* (`ward ops forgejo issue list-all`), never git files, so it never re-ingests `corpus/*.md`. inbox's own issues render to `corpus/coilysiren/inbox/*.md` (expected - inbox is a source), issues->files, no loop. Pushing files to inbox's git tree does not touch issue `updated_at`, so the incremental manifest is stable across runs. 4. **Credential + trust -> reuse the existing cache tier, unconditional, no role gate.** The container's system credential store (`/etc/ward-git-credentials`, coilyco-ops `FORGEJO_TOKEN`) already reads inbox - verified by cloning it this session. That is exactly how the existing private `cache` tier (`coilyco-bridge/*`) authenticates, so no new credential wiring. Mount it like any other cache-tier repo: **unconditional for every run that warms substrate**, honoring the existing `WARD_SUBSTRATE_SKIP` / context-level skips. Do **not** add a per-role gate - offline issue discovery is valuable to engineer, director, and advisor alike, and the corpus is already private so the trust surface matches what cache-tier repos already expose. ### Constraints honored - **Programmatic only, no LLM inference** - the renderer is a deterministic markdown dump of issue bodies + comments. Nothing here adds a model in the path. Kept. - **Discovery index, not a source of truth** - the rendered headers already carry the "confirm live via `ward ops forgejo issue view`" disclaimer. The mount changes nothing about that. A ward-side docs note should repeat the grep-to-locate / API-to-confirm discipline. - **Never public** - stays a `cache`-tier entry, never baked into the shareable dev-base image seed. ### The one real cost to flag The corpus grows **unbounded** - every closed issue, forever, in every container's `/substrate`. Today that is small. Left unbounded it is a slowly-growing scan-and-disk surface on every run. Not a blocker now, worth a **follow-up** to bound it (a recency window, or skip the corpus at `WARD_CONTEXT_LEVEL=0`). This is distinct from [#523](https://forgejo.coilysiren.me/coilyco-flight-deck/ward/issues/523)'s sparse-mount: the corpus is rendered *data*, not a code repo with a trifecta reference surface, so #523's trifecta-sparse pattern does not apply to it - a size bound would be its own mechanism. --- ## Implementation checklist, split by repo **agentic-os (producer, coordinated - do first, the ward mount is blocked on the corpus actually landing in inbox):** - [ ] `.forgejo/workflows/issue-corpus.yml`: set `MIRROR_REPO: coilysiren/inbox`. - [ ] Same workflow: after cloning the mirror, `mkdir -p "${MIRROR_DIR}/corpus"` and call `render-issue-corpus.py --mirror-dir "${MIRROR_DIR}/corpus"`. No renderer code change - the layout is `--mirror-dir`-controlled. `git add -A` from the clone root then commits `corpus/` and leaves `README.md` untouched. - [ ] Confirm the trufflehog scan still covers the rendered tree (the renderer scans its `--mirror-dir`, now `corpus/`) before the push. - [ ] Point the push credential (`ISSUE_CORPUS_PUSH_TOKEN`, or reuse the coilyco-ops write token) at **write access to `coilysiren/inbox`**. Retire the separate `coilysiren/issue-corpus` mirror repo and its dedicated provisioning - that rollout prerequisite is now gone. - [ ] Update `docs/issue-corpus.md`: MIRROR_REPO is inbox, output under `corpus/`, drop the "provision a new private mirror repo" bullet. **ward (consumer - this issue, after the corpus is live in inbox):** - [ ] Add `coilysiren/inbox cache` to `cmd/ward/containerassets/preclone-repos.txt` (cache tier - private, never image-baked). - [ ] Add the same line to the denormalized ansible copy, `infrastructure/scripts/clone-substrate-repos.sh` (the two stay denormalized per that file's own note). - [ ] No `warmSubstrateRepo` code change needed - the cache tier already does the credentialed private clone. Mount appears at `/substrate/inbox`, index at `/substrate/inbox/corpus/`. - [ ] Docs note (ward substrate docs) pointing agents at `/substrate/inbox/corpus/` as the greppable discovery index, repeating the confirm-live-via-`ward ops forgejo issue view` discipline. - [ ] `docs/FEATURES.md` entry if this counts as a user/agent-visible capability (offline issue-corpus grep). **follow-up (separate issue, do not fold in):** - [ ] Bound the corpus growth (recency window, or skip at `WARD_CONTEXT_LEVEL=0`) once its size warrants it. --- Design pass by the read-only director surface session (claude-linux-cbbb76a95881-cb37-she-her), in-session substitute for the advisor run blocked by #577. Ready for an engineer dispatch once the agentic-os producer change lands the corpus in inbox.
Author
Owner

Blocked-by: the producer half is now tracked at agentic-os#324 - repoint the renderer's MIRROR_REPO at coilysiren/inbox and render under corpus/. This ward mount should not merge until that lands the corpus in inbox (else the cache-tier clone is an empty tree). Both are ready for an engineer, producer first.

**Blocked-by:** the producer half is now tracked at [agentic-os#324](https://forgejo.coilysiren.me/coilyco-flight-deck/agentic-os/issues/324) - repoint the renderer's `MIRROR_REPO` at `coilysiren/inbox` and render under `corpus/`. This ward mount should not merge until that lands the corpus in inbox (else the cache-tier clone is an empty tree). Both are ready for an engineer, producer first.
Member

Superseded by Kai on 2026-07-08: do not mount the git-rendered issue corpus into /substrate via coilysiren/inbox. The newer coilyco-bridge/deploy forgejo-issues service is the right home for this: it already syncs Forgejo issues into Postgres and exposes an Airtable-ish NocoDB view. The agent offline-reference requirement should be absorbed there instead of carrying this substrate/inbox corpus path forward.

Keeping this issue open would preserve the first, now-unwanted design. Closing it as superseded.

Superseded by Kai on 2026-07-08: do not mount the git-rendered issue corpus into `/substrate` via `coilysiren/inbox`. The newer `coilyco-bridge/deploy` `forgejo-issues` service is the right home for this: it already syncs Forgejo issues into Postgres and exposes an Airtable-ish NocoDB view. The agent offline-reference requirement should be absorbed there instead of carrying this substrate/inbox corpus path forward. Keeping this issue open would preserve the first, now-unwanted design. Closing it as superseded.
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
coilyco-flight-deck/ward#575
No description provided.