coilyco-gaming/eco-app
2
0
Fork 0
Code Issues 16 Pull requests Projects Releases Packages Wiki Activity Actions

EcoReplay fails to load on kai-server: DllNotFoundException e_sqlite3 (native lib not on the .NET single-file search path) #71

New issue
Closed
opened 2026-07-06 04:32:06 +00:00 by coilysiren · 2 comments
coilysiren commented 2026-07-06 04:32:06 +00:00
Owner
Copy link

What happened

Deploying EcoReplay to the live kai-server Eco install (2026-07-06 deploy session) crashed the server on boot:

Loading SQLitePCLRaw.core... / batteries_v2 / Microsoft.Data.Sqlite / provider.e_sqlite3
Loading EcoReplay...
Initializing EcoReplayPlugin ... Finished in 0.020 sec
Failed to start the server. Exception was Exception: DllNotFoundException
Unable to load shared library 'e_sqlite3' or one of its dependencies.
/home/kai/.net/EcoServer/FMyt7Bcg8UHB/libe_sqlite3.so: cannot open shared object file: No such file or directory
   at SQLitePCL.SQLite3Provider_e_sqlite3.NativeMethods.sqlite3_libversion_number()
   at SQLitePCL.Batteries_V2.Init()
The type initializer for 'Microsoft.Data.Sqlite.SqliteConnection' threw an exception.

service_active=0 - hard boot failure, server down/crash-looping until EcoReplay was removed. Rolled back; server healthy. The mod itself loaded and initialized fine (the plugin init ran); it fails the moment it opens the SQLite connection.

Root cause

The Eco server runs as a .NET single-file bundle, so the native-lib resolver for DllImport("e_sqlite3") searches only the bundle extraction dir (~/.net/EcoServer/<hash>/), NOT the mod folder or its runtimes/linux-x64/native/ path. Staging libe_sqlite3.so in Mods/EcoReplay/ (both flattened and under runtimes/linux-x64/native/) does not put it on the search path - confirmed by the probed paths in the exception (all under the extraction dir).

Recommended fix (self-contained in the mod, no infra change)

In EcoReplayPlugin init, register a NativeLibrary.SetDllImportResolver on the SQLitePCLRaw provider assembly that resolves e_sqlite3 to the mod's own directory:

NativeLibrary.SetDllImportResolver(
    typeof(SQLitePCL.SQLite3Provider_e_sqlite3).Assembly,
    (name, asm, searchPath) => name == "e_sqlite3"
        ? NativeLibrary.Load(Path.Combine(Path.GetDirectoryName(typeof(EcoReplayPlugin).Assembly.Location)!, "libe_sqlite3.so"))
        : IntPtr.Zero);

Register it before the first SqliteConnection/Batteries_V2.Init(). Alternative (heavier, fleet-level): set LD_LIBRARY_PATH to the mod native dir in the eco-server systemd unit / eco-server-start.sh (infrastructure repo).

Packaging (already staged, verified)

Mods/EcoReplay/ needs: EcoReplay.dll, EcoReplay.deps.json, Microsoft.Data.Sqlite.dll, SQLitePCLRaw.{core,batteries_v2,provider.e_sqlite3}.dll, libe_sqlite3.so (linux-x64). Staged bundle is at ~/eco-staging/EcoReplay/ on kai-server.

Validation

Restart-gated - live validation lands in the next restart window, not tonight. A headless engineer can implement + build green now. Refs: eco-app#67.

## What happened Deploying EcoReplay to the live kai-server Eco install (2026-07-06 deploy session) crashed the server on boot: ``` Loading SQLitePCLRaw.core... / batteries_v2 / Microsoft.Data.Sqlite / provider.e_sqlite3 Loading EcoReplay... Initializing EcoReplayPlugin ... Finished in 0.020 sec Failed to start the server. Exception was Exception: DllNotFoundException Unable to load shared library 'e_sqlite3' or one of its dependencies. /home/kai/.net/EcoServer/FMyt7Bcg8UHB/libe_sqlite3.so: cannot open shared object file: No such file or directory at SQLitePCL.SQLite3Provider_e_sqlite3.NativeMethods.sqlite3_libversion_number() at SQLitePCL.Batteries_V2.Init() The type initializer for 'Microsoft.Data.Sqlite.SqliteConnection' threw an exception. ``` `service_active=0` - hard boot failure, server down/crash-looping until EcoReplay was removed. Rolled back; server healthy. The mod itself loaded and initialized fine (the plugin init ran); it fails the moment it opens the SQLite connection. ## Root cause The Eco server runs as a **.NET single-file bundle**, so the native-lib resolver for `DllImport("e_sqlite3")` searches only the bundle **extraction dir** (`~/.net/EcoServer/<hash>/`), NOT the mod folder or its `runtimes/linux-x64/native/` path. Staging `libe_sqlite3.so` in `Mods/EcoReplay/` (both flattened and under `runtimes/linux-x64/native/`) does not put it on the search path - confirmed by the probed paths in the exception (all under the extraction dir). ## Recommended fix (self-contained in the mod, no infra change) In `EcoReplayPlugin` init, register a `NativeLibrary.SetDllImportResolver` on the SQLitePCLRaw provider assembly that resolves `e_sqlite3` to the mod's own directory: ```csharp NativeLibrary.SetDllImportResolver( typeof(SQLitePCL.SQLite3Provider_e_sqlite3).Assembly, (name, asm, searchPath) => name == "e_sqlite3" ? NativeLibrary.Load(Path.Combine(Path.GetDirectoryName(typeof(EcoReplayPlugin).Assembly.Location)!, "libe_sqlite3.so")) : IntPtr.Zero); ``` Register it before the first `SqliteConnection`/`Batteries_V2.Init()`. Alternative (heavier, fleet-level): set `LD_LIBRARY_PATH` to the mod native dir in the eco-server systemd unit / `eco-server-start.sh` (infrastructure repo). ## Packaging (already staged, verified) `Mods/EcoReplay/` needs: `EcoReplay.dll`, `EcoReplay.deps.json`, `Microsoft.Data.Sqlite.dll`, `SQLitePCLRaw.{core,batteries_v2,provider.e_sqlite3}.dll`, `libe_sqlite3.so` (linux-x64). Staged bundle is at `~/eco-staging/EcoReplay/` on kai-server. ## Validation Restart-gated - live validation lands in the next restart window, not tonight. A headless engineer can implement + build green now. Refs: eco-app#67.
coilyco-ops commented 2026-07-06 04:32:59 +00:00
Member
Copy link

🔒 Reserved by ward agent --driver claude — container engineer-claude-eco-app-71 on host KAI-DESKTOP-TOWER is carrying this issue (reserved 2026-07-06T04:32:55Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); --force overrides.

Do not comment on or edit this issue to steer the run while it is reserved. The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a new issue, dispatched fresh — that is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).

run seed context — what this run is carrying (ward#609)
  • Resolved: coilyco-gaming/eco-app#71 · branch issue-71 · driver claude · workflow direct-main
  • Run: engineer-claude-eco-app-71 · ward v0.413.0 · dispatched 2026-07-06T04:32:55Z
  • Comment thread: 0 included in the pre-flight read, 0 stripped (ward's own automated comments).

Issue body as seeded:

## What happened
Deploying EcoReplay to the live kai-server Eco install (2026-07-06 deploy session) crashed the server on boot:

` ` `
Loading SQLitePCLRaw.core... / batteries_v2 / Microsoft.Data.Sqlite / provider.e_sqlite3
Loading EcoReplay...
Initializing EcoReplayPlugin ... Finished in 0.020 sec
Failed to start the server. Exception was Exception: DllNotFoundException
Unable to load shared library 'e_sqlite3' or one of its dependencies.
/home/kai/.net/EcoServer/FMyt7Bcg8UHB/libe_sqlite3.so: cannot open shared object file: No such file or directory
   at SQLitePCL.SQLite3Provider_e_sqlite3.NativeMethods.sqlite3_libversion_number()
   at SQLitePCL.Batteries_V2.Init()
The type initializer for 'Microsoft.Data.Sqlite.SqliteConnection' threw an exception.
` ` `

`service_active=0` - hard boot failure, server down/crash-looping until EcoReplay was removed. Rolled back; server healthy. The mod itself loaded and initialized fine (the plugin init ran); it fails the moment it opens the SQLite connection.

## Root cause
The Eco server runs as a **.NET single-file bundle**, so the native-lib resolver for `DllImport("e_sqlite3")` searches only the bundle **extraction dir** (`~/.net/EcoServer/<hash>/`), NOT the mod folder or its `runtimes/linux-x64/native/` path. Staging `libe_sqlite3.so` in `Mods/EcoReplay/` (both flattened and under `runtimes/linux-x64/native/`) does not put it on the search path - confirmed by the probed paths in the exception (all under the extraction dir).

## Recommended fix (self-contained in the mod, no infra change)
In `EcoReplayPlugin` init, register a `NativeLibrary.SetDllImportResolver` on the SQLitePCLRaw provider assembly that resolves `e_sqlite3` to the mod's own directory:

` ` `csharp
NativeLibrary.SetDllImportResolver(
    typeof(SQLitePCL.SQLite3Provider_e_sqlite3).Assembly,
    (name, asm, searchPath) => name == "e_sqlite3"
        ? NativeLibrary.Load(Path.Combine(Path.GetDirectoryName(typeof(EcoReplayPlugin).Assembly.Location)!, "libe_sqlite3.so

… (truncated to 2000 chars; full body is on this issue)

Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.413.0).

— Claude (she/her), via ward agent

<!-- ward-agent-reservation --> 🔒 Reserved by `ward agent --driver claude` — container `engineer-claude-eco-app-71` on host `KAI-DESKTOP-TOWER` is carrying this issue (reserved 2026-07-06T04:32:55Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); `--force` overrides. **Do not comment on or edit this issue to steer the run while it is reserved.** The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a **new issue, dispatched fresh** — that is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494). <details><summary>run seed context — what this run is carrying (ward#609)</summary> - **Resolved:** `coilyco-gaming/eco-app#71` · branch `issue-71` · driver `claude` · workflow `direct-main` - **Run:** `engineer-claude-eco-app-71` · ward `v0.413.0` · dispatched `2026-07-06T04:32:55Z` - **Comment thread:** 0 included in the pre-flight read, 0 stripped (ward's own automated comments). **Issue body as seeded:** ``` ## What happened Deploying EcoReplay to the live kai-server Eco install (2026-07-06 deploy session) crashed the server on boot: ` ` ` Loading SQLitePCLRaw.core... / batteries_v2 / Microsoft.Data.Sqlite / provider.e_sqlite3 Loading EcoReplay... Initializing EcoReplayPlugin ... Finished in 0.020 sec Failed to start the server. Exception was Exception: DllNotFoundException Unable to load shared library 'e_sqlite3' or one of its dependencies. /home/kai/.net/EcoServer/FMyt7Bcg8UHB/libe_sqlite3.so: cannot open shared object file: No such file or directory at SQLitePCL.SQLite3Provider_e_sqlite3.NativeMethods.sqlite3_libversion_number() at SQLitePCL.Batteries_V2.Init() The type initializer for 'Microsoft.Data.Sqlite.SqliteConnection' threw an exception. ` ` ` `service_active=0` - hard boot failure, server down/crash-looping until EcoReplay was removed. Rolled back; server healthy. The mod itself loaded and initialized fine (the plugin init ran); it fails the moment it opens the SQLite connection. ## Root cause The Eco server runs as a **.NET single-file bundle**, so the native-lib resolver for `DllImport("e_sqlite3")` searches only the bundle **extraction dir** (`~/.net/EcoServer/<hash>/`), NOT the mod folder or its `runtimes/linux-x64/native/` path. Staging `libe_sqlite3.so` in `Mods/EcoReplay/` (both flattened and under `runtimes/linux-x64/native/`) does not put it on the search path - confirmed by the probed paths in the exception (all under the extraction dir). ## Recommended fix (self-contained in the mod, no infra change) In `EcoReplayPlugin` init, register a `NativeLibrary.SetDllImportResolver` on the SQLitePCLRaw provider assembly that resolves `e_sqlite3` to the mod's own directory: ` ` `csharp NativeLibrary.SetDllImportResolver( typeof(SQLitePCL.SQLite3Provider_e_sqlite3).Assembly, (name, asm, searchPath) => name == "e_sqlite3" ? NativeLibrary.Load(Path.Combine(Path.GetDirectoryName(typeof(EcoReplayPlugin).Assembly.Location)!, "libe_sqlite3.so ``` … (truncated to 2000 chars; full body is on this issue) Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.413.0). </details> <!-- ward-agent-signature --> — Claude (she/her), via `ward agent`
coilysiren commented 2026-07-06 04:38:06 +00:00
Author
Owner
Copy link

WARD-OUTCOME: done - EcoReplay's e_sqlite3 resolver now registers on the actual provider assembly, landed on main (ee6d1b8).

This one was a nice "the fix was already there, but pointed one assembly to the left" bug. A SetDllImportResolver was already installed, but on SQLitePCLRaw.core plus a runtime AppDomain.GetAssemblies() scan for the provider assembly - and that scan runs at plugin-init before Batteries_V2.Init() has loaded SQLitePCLRaw.provider.e_sqlite3, so it always came up empty and the resolver never got attached to the assembly whose DllImport("e_sqlite3") actually fires. Swapping to a hard typeof(SQLite3Provider_e_sqlite3).Assembly both force-loads the provider and targets the right assembly. Also added the runtimes/linux-x64/native/ probe path so either staging layout resolves.

Build is green (the provider typeof compiles, confirming reachability) and the 9 mod unit tests pass. Confidence is high on the registration logic, but this is restart-gated - I can't exercise a live single-file bundle here, so the real proof is the next kai-server restart window actually booting with EcoReplay in. Worth watching that boot; if it still probes only the extraction dir, the fallback is the LD_LIBRARY_PATH route in the systemd unit (infra repo). No follow-ups filed yet - will hold until live validation confirms.

WARD-OUTCOME: done - EcoReplay's e_sqlite3 resolver now registers on the actual provider assembly, landed on main (ee6d1b8). This one was a nice "the fix was already there, but pointed one assembly to the left" bug. A `SetDllImportResolver` was already installed, but on `SQLitePCLRaw.core` plus a runtime `AppDomain.GetAssemblies()` scan for the provider assembly - and that scan runs at plugin-init before `Batteries_V2.Init()` has loaded `SQLitePCLRaw.provider.e_sqlite3`, so it always came up empty and the resolver never got attached to the assembly whose `DllImport("e_sqlite3")` actually fires. Swapping to a hard `typeof(SQLite3Provider_e_sqlite3).Assembly` both force-loads the provider and targets the right assembly. Also added the `runtimes/linux-x64/native/` probe path so either staging layout resolves. Build is green (the provider typeof compiles, confirming reachability) and the 9 mod unit tests pass. Confidence is high on the *registration* logic, but this is restart-gated - I can't exercise a live single-file bundle here, so the real proof is the next kai-server restart window actually booting with EcoReplay in. Worth watching that boot; if it still probes only the extraction dir, the fallback is the `LD_LIBRARY_PATH` route in the systemd unit (infra repo). No follow-ups filed yet - will hold until live validation confirms.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
issue-109
issue-106
ward-salvage/eco-app-a58b14ad
issue-104
No results found.
Labels
Clear labels   
burndown-2026-06
Backlog burndown June 2026

  
headless
Safe for autonomous warded engineer carry

  
consult
'A human decision, design, or external action must happen first. Fail-closed default for unlabeled/low-confidence.

  
headless
Agent can take it from open issue to merged change with no human in the loop. The auto-burndown queue.

  
interactive
Agent does the work but pauses at a human checkpoint mid-flight (design choice, destructive step, human-only verification).

  
P0
priority tier

  
P1
priority tier

  
P2
priority tier

  
P3
priority tier

  
P4
priority tier

No labels
burndown-2026-06
headless
consult
headless
interactive
P0
P1
P2
P3
P4
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
coilyco-gaming/eco-app#71
No description provided.