design: replace sqlite3 in the eco replay with a C#-on-Linux-amenable DB (broke the deploy) #88
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Incident + direction (Kai)
Last night the eco replay deploy broke on a sqlite3 Linux compat issue. Kai's steer: "why don't we just use whatever database type is most amenable to C# on Linux? we've discovered that C# is not the answer."
This is a design brief, not an implementation ticket - investigate and post a concrete recommendation as a comment for Kai to approve. Do not migrate anything yet.
The ambiguity to resolve (flag it explicitly in your answer)
"C# is not the answer" reads two ways, and the recommendation differs sharply:
e_sqlite3.sothat is not present or ABI-mismatched in the deploy container). Fix = pick a DB with zero native-lib dependency so it "just works" under .NET on Linux.Diagnose which one Kai is actually hitting, with evidence from the code + the deploy break, and recommend accordingly.
Investigate
services/eco-app/**bundle in coilyco-bridge/deploy, or a sibling).:memory:path issue, or a driver version? Pin the actual failure, not a guess.SQLitePCLRaw.bundle_e_sqlite3ships the Linux native; sometimes the break is just a missing bundle package. Cheapest if the replay is otherwise fine.Deliverable
A comment with: the pinned root cause, the resolved read on the C# ambiguity, a recommended persistence choice (+ a short migration sketch and blast radius), and the follow-up implementation issue(s) to file. If the answer is "just add the SQLitePCLRaw bundle," say so plainly - the cheapest correct fix wins.
🔎 ward agent advisor
ward agent advisor --driver clauderan a one-shot standard research pass on this question:Diagnosis: this is native-SQLite interop on the Linux Eco server, not a reason to leave C#
Where the eco replay actually lives
The replay is two pieces in this repo (
coilyco-gaming/eco-app), confirmed indocs/replay/README.md:mods/replay/src/) —EcoReplayPluginimplementsIGameActionAware, receives everyGameAction, and appends to SQLite viaMicrosoft.Data.Sqlite(EventStore.cs). It runs in-process inside the live Eco game server and is shipped out of band — it is explicitly not in the eco-app Docker image (.forgejo/workflows/build-publish.ymlmods:job comment: "shipped to the game server OUT OF BAND … NOT in the eco-app image").src/eco_replay/main.py) — a thin FastAPI re-server mounted at/replay/apiin the fused service. It reads the mod's data one of three ways: the SQLite file directly via stdlibsqlite3(ECO_REPLAY_DB), the mod's HTTP endpoint (UPSTREAM_URL), or mock data.The break is on the C# side. The eco-app container is
python:3.13(Dockerfile) — stdlibsqlite3there is statically linked, has no native-lib gap, and did not break. The thing that "broke the deploy" is the mod loadinglibe_sqlite3.soinside the Eco server.Pinned root cause
This is a recurrence of eco-app#71, not a new class of bug. The evidence chain:
Microsoft.Data.Sqlite→SQLitePCLRaw.bundle_e_sqlite3needs a nativelibe_sqlite3.sodlopen'd at first use (Batteries_V2.Init).AssemblyLoadContext.Default.LoadFromStream, and the Eco server ships as a single-file .NET bundle, so the default native resolver searches only the bundle extraction dir (~/.net/EcoServer/<hash>/), never the mod folder or itsruntimes/linux-x64/native/. #71's fix was an elaborateDllImportResolverregistered on the provider assembly (SQLite3Provider_e_sqlite3) that probesmodDirandmodDir/runtimes/linux-x64/native/for the.so(seeEcoReplayPlugin.InstallNativeLibraryResolver, and the #71 commitc2bef47)..sois physically present in one of those probed paths. The most likely #88 trigger: the out-of-band ship copiedEcoReplay.dll(and its managed deps) but not theruntimes/linux-x64/native/libe_sqlite3.sosibling thatdotnet buildemits underbin/Release/net10.0/. With the file absent, everyFile.Existscandidate misses, the resolver returnsIntPtr.Zero, default resolution runs,dlopenfails →DllNotFoundException→ crash-loop on server start (exactly the #71 failure mode). A secondary candidate is an Eco server runtime bump changing the bundle extraction layout and re-defeating the resolver — same root class either way.This is a moving target. #71 already patched the interop once; #88 is the second break. Chasing a native
.sostaged into a mod folder that gets reloaded throughLoadFromStreaminto a single-file bundle is fundamentally fragile, and every Eco server update can re-break it. That is the real signal in Kai's steer.Resolving the "C# is not the answer" ambiguity → it's reading (i)
Interpretation (ii) — "reconsider C# for the replay entirely" — is not on the table for the recorder. The recorder must be a C# Eco plugin: it implements
IGameActionAwareand receivesGameActionobjects in-process inside the game server. No other language can subscribe to that firehose — there is no non-C# way to write this half. The Python half already is the non-C# piece, and it's fine.So Kai is hitting (i): the native-DB path is the dead end, the mod stays C#. "C# is not the answer" means don't make a C# mod drive a native-code SQLite inside the Eco bundle — not rewrite the recorder. The fix is to pick a persistence layer with zero native dependency.
Weighing the options (no-native-lib lens)
bundle_e_sqlite3. If the only defect is that the ship step dropsruntimes/linux-x64/native/libe_sqlite3.so, fixing the ship manifest unblocks the deploy today. But it is the exact band-aid Kai just rejected: it keeps the native dependency and the second break proves it's not durable. Use only as an emergency hotfix, not the strategy.ECO_REPLAY_DBneeds the app container to see the game server's filesystem). It keeps indexed queries (filter by citizen/type/since,ORDER BY id DESC LIMIT ≤1000) at any volume. Cost: the game server now needs network + a credential to Postgres, a Postgres outage becomes a write path failure (the store already tolerates + drops write errors, so this degrades gracefully), and it needs a matching change incoilyco-bridge/deploy(a DB + ExternalSecret). Strong default if replay stays a real feature.Storage/backup loop for free" property. But it breaks the Python direct-file read contract (src/eco_replay/main.pyopens the file with stdlibsqlite3; LiteDB's BSON format is unreadable bysqlite3, and there's no solid Python LiteDB reader). Choosing LiteDB forces the Python side onto theUPSTREAM_URLHTTP path only. Workable, but it trades one interop headache for a read-contract rewrite with less upside than Postgres.Newtonsoft.Json; writing newline-delimited JSON toStorage/EcoReplay.jsonlis zero-dependency, keeps the single-file-rides-backup property, and is trivially readable by both the mod's own HTTP endpoint and the Python reader (swapsqlite3for line reads). The truest expression of "pick whatever is most amenable to C#-on-Linux" — the most amenable DB is no native DB. Cost: no indexes, so a query is a reverse file scan; fine at homelab volume withLIMIT ≤1000, but degrades if the event log grows large.The "most pages are just data, only world/region is visual" simplification matters here
That sibling steer de-emphasizes replay data and argues against standing up dedicated Postgres infra for it. If replay is shrinking toward a low-traffic, possibly-optional surface, the JSONL path is the proportionate answer; if replay stays a first-class queryable feature, Postgres is worth the deploy wiring.
Recommendation
Drop
Microsoft.Data.Sqlitefrom the mod. Pick by whether replay stays a real feature:sqlite3read contract for no net gain over the two above) and reject "just re-add the bundle" as the strategy (it's the band-aid #71 already tried; offer it only as a same-day hotfix if the deploy must be unblocked before the swap lands).Kai's call to make: first-class replay (→ Postgres) vs de-emphasized replay (→ JSONL). I lean JSONL given the "most pages are just data" steer plus the fact that the recorder's needs are genuinely just append + bounded filtered reads — but Postgres is the right answer the moment replay wants real query scale.
Migration sketch + blast radius
Both options are contained to
coilyco-gaming/eco-app(the mod and its Python reader live in the same repo), plus a deploy-repo change only if Postgres is chosen.EventStore.csto append serialized rows toStorage/EcoReplay.jsonl(keep the existing bounded-channel + background-writer design — it maps cleanly to batched line appends; keepQuery()as a bounded reverse scan). Updatesrc/eco_replay/main.py_fetch_from_db/_stats_from_dbto read the JSONL file instead ofsqlite3. Drop theMicrosoft.Data.SqlitePackageReference and the entireInstallNativeLibraryResolverblock. No deploy-repo change (no new infra). Updatedocs/replay/README.mdanddocs/FEATURES.md.EventStore.cstoNpgsql(schema is the sameeventstable; the batched-transaction writer ports directly), take a connection string from mod config/env. Pointsrc/eco_replay/main.pyat Postgres (swapsqlite3forpsycopg, or read exclusively viaUPSTREAM_URL). Matching change incoilyco-bridge/deploy(services/eco-app/**): a Postgres instance/DB + an ExternalSecret for the credential, per the AGENTS.md cross-repo contract. Update the docs.Follow-up implementation issues to file (after Kai approves a direction — do NOT migrate yet)
All are in
coilyco-gaming/eco-appunless noted; I'm holding them until Kai picks JSONL vs Postgres so we don't file work for the rejected branch:coilyco-gaming/eco-app: swap the replay mod off native SQLite to . RewriteEventStore.cs, dropMicrosoft.Data.Sqlite+ the native resolver, keep the bounded-channel writer. Depends on Kai's direction; unblocks #2 and #3.coilyco-gaming/eco-app: point the Python reader (src/eco_replay/main.py) at the new store. Depends on #1's storage format being locked.coilyco-bridge/deploy: provision Postgres + ExternalSecret for eco-app replay underservices/eco-app/**. Depends on #1 defining the connection contract; can proceed in parallel once the schema/DSN shape is fixed.coilyco-gaming/eco-app: harden the out-of-band mod ship step so a mod'sruntimes/**natives can't be silently dropped — captures the immediate #88 trigger regardless of which store wins, and de-risks any future native dep.If the deploy must be unblocked today before the swap lands, the emergency hotfix is: ensure
runtimes/linux-x64/native/libe_sqlite3.sois actually staged next toEcoReplay.dllon the game server (the path #71's resolver probes). That is the cheapest correct stopgap — but it is a stopgap, and the durable answer is dropping the native dependency.Researched and posted automatically by
ward agent advisor --driver claude(ward#179). This is one-shot research, not a carried change - verify before acting on it.— Claude (she/her), via
ward agent