Slice 3: caching layer for remote-first Forgejo ingest #110

New issue

Closed

opened 2026-06-17 11:59:08 +00:00 by coilysiren · 1 comment

coilysiren commented

2026-06-17 11:59:08 +00:00

Owner

Slice 3 (deferred from #109): caching layer for remote-first ingest

Remote-first Forgejo ingest (#109, Slices 1+2) is live: REPO_RECALL_REMOTE_FIRST seeds
targets from fetch_active_repos, and issues/PRs/milestones/commits hydrate with no local
clone. Caching was explicitly fenced as Slice 3.

Want: persist the last-good remote snapshot (issues/PRs/milestones/commits) across refreshes
and process restarts so a cold container shows data immediately and a transient Forgejo blip
doesn't blank the dashboard. Today a fresh cache dir is empty until the first full remote pass
completes (~50s for ~33 repos).

Notes / adjacent cleanups uncovered during #109:

finalize_repo_aggregates runs in the blocking phase before the remote pass, so
remote-ingested commits needed a second finalize call after ingest_remote_commits
(already done in #109). A cache layer should make aggregate recompute ordering explicit.
The per-source scheduler can run git_log (which wipe_git_logs the commit tables) without
github_remote in the same tick; in remote-first that transiently drops remote commits until
the next remote pass. A cache / source-ownership split would fix this cleanly.

refs #109

## Slice 3 (deferred from #109): caching layer for remote-first ingest Remote-first Forgejo ingest (#109, Slices 1+2) is live: `REPO_RECALL_REMOTE_FIRST` seeds targets from `fetch_active_repos`, and issues/PRs/milestones/commits hydrate with no local clone. Caching was explicitly fenced as Slice 3. **Want:** persist the last-good remote snapshot (issues/PRs/milestones/commits) across refreshes and process restarts so a cold container shows data immediately and a transient Forgejo blip doesn't blank the dashboard. Today a fresh cache dir is empty until the first full remote pass completes (~50s for ~33 repos). **Notes / adjacent cleanups uncovered during #109:** - `finalize_repo_aggregates` runs in the blocking phase *before* the remote pass, so remote-ingested commits needed a second `finalize` call after `ingest_remote_commits` (already done in #109). A cache layer should make aggregate recompute ordering explicit. - The per-source scheduler can run `git_log` (which `wipe_git_log`s the commit tables) without `github_remote` in the same tick; in remote-first that transiently drops remote commits until the next remote pass. A cache / source-ownership split would fix this cleanly. refs https://forgejo.coilysiren.me/coilyco-flight-deck/repo-recall/issues/109

coilysiren referenced this issue

2026-06-17 11:59:24 +00:00

Remote-first refinements: target-limit ordering + git_log wipe interaction #112

coilysiren referenced this issue from a commit

2026-06-17 12:03:23 +00:00

feat(forgejo): remote commit ingest from the commits API

coilysiren commented

2026-06-17 18:52:18 +00:00

Author

Owner

Closing. The Claude-session observability tree (repo-recall / luca / session-lattice) is retired in the June 2026 surface reduction.

coilysiren closed this issue