Replace replay native SQLite with bounded JSONL storage #159

New issue

Closed

opened 2026-07-26 00:21:50 +00:00 by coilyco-ops · 0 comments

coilyco-ops commented

2026-07-26 00:21:50 +00:00

Member

Outcome

Replace the replay mod's native SQLite dependency with one append-only JSONL store that both the C# recorder and Python reader can consume on Linux without native-library loading.

Decision source: #88.

Scope

Replace mods/replay/src/EventStore.cs persistence with newline-delimited Storage/EcoReplay.jsonl.
Preserve the bounded channel and background writer so game-thread work remains non-blocking.
Preserve the current event schema and bounded filtered query contract.
Implement bounded retention or compaction for the newest 2,000,000 rows without deleting the existing SQLite database.
Update the Python direct-file reader to consume JSONL. Keep the mod HTTP upstream path unchanged.
Remove Microsoft.Data.Sqlite, SQLitePCLRaw runtime assets, and the native-library resolver from the replay mod.
Add C# and Python coverage for append, malformed or partial final lines, filters, limits, retention, and restart recovery.
Update docs/replay/README.md and docs/FEATURES.md.

Safety and migration

The implementation must not delete, rewrite, or silently convert the existing EcoReplay.db. It remains a recoverable historical archive. Any one-time import of old rows is a separate operator-approved task because deduplication and backup boundaries need live evidence.

Validation

ward exec test
ward exec lint
ward exec build-mod-replay
ward exec test-mod-replay
ward exec precommit

Non-goals

No live mod deployment, server restart, Postgres provisioning, destructive SQLite cleanup, or production verification.

## Outcome Replace the replay mod's native SQLite dependency with one append-only JSONL store that both the C# recorder and Python reader can consume on Linux without native-library loading. Decision source: #88. ## Scope * Replace `mods/replay/src/EventStore.cs` persistence with newline-delimited `Storage/EcoReplay.jsonl`. * Preserve the bounded channel and background writer so game-thread work remains non-blocking. * Preserve the current event schema and bounded filtered query contract. * Implement bounded retention or compaction for the newest 2,000,000 rows without deleting the existing SQLite database. * Update the Python direct-file reader to consume JSONL. Keep the mod HTTP upstream path unchanged. * Remove `Microsoft.Data.Sqlite`, SQLitePCLRaw runtime assets, and the native-library resolver from the replay mod. * Add C# and Python coverage for append, malformed or partial final lines, filters, limits, retention, and restart recovery. * Update `docs/replay/README.md` and `docs/FEATURES.md`. ## Safety and migration The implementation must not delete, rewrite, or silently convert the existing `EcoReplay.db`. It remains a recoverable historical archive. Any one-time import of old rows is a separate operator-approved task because deduplication and backup boundaries need live evidence. ## Validation * `ward exec test` * `ward exec lint` * `ward exec build-mod-replay` * `ward exec test-mod-replay` * `ward exec precommit` ## Non-goals No live mod deployment, server restart, Postgres provisioning, destructive SQLite cleanup, or production verification.

coilyco-ops added the

headless

autonomy

headless

labels

2026-07-26 00:21:51 +00:00

coilyco-ops referenced this issue

2026-07-26 00:21:51 +00:00

design: replace sqlite3 in the eco replay with a C#-on-Linux-amenable DB (broke the deploy) #88

coilyco-ops referenced this issue

2026-07-26 00:38:02 +00:00

[interactive] Resolve remaining Eco App decisions and live checkpoints #155