Tracking: 2026-08-12 full-surface QA sweep of the eco MCP (22 tools, 23 findings) #240
Labels
No labels
burndown-2026-06
headless
autonomy
async-consult
autonomy
epic
autonomy
headless
autonomy
live-collab
c#
priority
P0
priority
P1
priority
P2
priority
P3
priority
P4
role/ai
role/creator
role/design
role/director
role/engineer
role/exec
role/human
role/ops
role/qa
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
coilyco-gaming/eco-app#240
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Tracking issue for a full adversarial sweep of all 22 eco MCP tools against Eco via Sirens (
eco.coilysiren.me:3001), cycle 14 / day 40, Eco 0.13.0.4, on 2026-08-12. Cross-server probes against GreenLeaf Prime and one unreachable host.Method: every tool called at least once, every filter parameter exercised, every stateful verb round-tripped, and each tool's answers cross-checked against every other tool's answer to the same question. Most findings below were only visible from that cross-check — no single tool used alone would surface the trade-total or
byCitizendiscrepancies.Root-cause clustering and suggested fix order are in the comments below. Short version: 23 findings collapse into four families plus a tail of six one-offs.
Findings index
Family 1 — currency never parsed out of the trade ledger
get_stores.currencies[]empty on stores with 4.6k volumefair_pricein-game side entirely nullget_marketreturnsmarkets: []for every queryFamily 2 — event-log rows get no entity resolution
subject/settlementblanktop_stations: [{"(unknown)": 6858.75}]Family 3 — no response-size policy (detail below)
get_currencyfilter never narrowsget_progressionper-citizen timelinesFamily 4 — derived and narrative fields computed loosely
byCitizensums quantities under an event-shaped namesettlementsFoundedadds foundings to foundation placementstotalCulture: 0republished as a KPITail — one-offs
get_speciesclaims a missing admin key that is presentget_mapnegative coords at the world seamexplain_itemignorescategory, alwaysfacts: []get_government.scopenames one settlement, returns fivetrade_watcherscreate summary discards the supplied labelFamily 3 detail: response size
Retained here because the three Family 3 issues reference these measurements.
get_speciesget_progressionget_tradesget_storesget_currencyexplain_itemget_crafting_atlasget_regionget_mapget_civics7 of 22 tools cannot return a usable answer to an MCP client. The caller gets an error plus a spill file and has to re-query it out of band, which defeats the point of the tool.
No tool in the set takes a
limit,page,top_n, orsinceparameter. The three filters that exist (item,currency,server) either do not reduce payload (#231) or return nothing (#218). Every large response is "everything, always":get_trades— all 526 detailed rows plus every aggregateget_stores— 111 stores and 43 traders with fulltopItems/topCounterpartiesget_progression— per-citizen event timelines for 80 citizensget_region— all 94 species including the 78 marked stableget_civics— every demographic row, duplicates includedDirection:
limit/top_non every list-returning tool, with a default a client can consume.get_currencyholders note (185 copies, ~37 KB) and theget_mapcolour tables.What the sweep did not find
Recorded so the finding count is not read as "the pipeline is unsound". Every arithmetic check against the raw layer passed:
get_world— nineperActionCountssum exactly tototalEvents(13,333)get_crafting_atlas— four sum exactly tototalEvents(18,092)DigOrMine3,129 andChopTree1,331 agree across two independently-computed toolsget_civicsturnout: 90 / (90 + 218) = 0.2922 ✓get_trades—rollupTrades+detailedTrades=totalTrades✓trade_watchers— fullcreate→list→evaluate→removelifecycle correct, predicate correct, no residueNo crashes, no security findings, no data loss. This is an accuracy-and-trust problem in the presentation layer, not an ingest problem.
Root-cause clustering for the 2026-08-12 sweep (#217–#239)
Note on scope: this issue was filed for payload size specifically. Parking the full clustering here because it is the closest thing the sweep has to a tracking issue — retitle or move if you'd rather keep this one narrow.
24 issues came out of the sweep, but they are not 24 independent defects. They collapse into four families plus a tail of one-offs. Suggested fix order is by how much each family unblocks.
1. Currency is never parsed out of the trade ledger
get_tradesreturnscurrency: ""on all 526 detailed rows while settingtrade_currency_column_seen: true. Everything downstream that needs a currency name then fails in its own way.get_stores.currencies[]empty on stores with 4.6k volumefair_pricein-game side null (partly; also depends on family 2 of #218)get_marketempty (unconfirmed dependency, but same data path)One fix, up to four issues. Start here.
find_traderesolves currency correctly from live store data (Spectres, Racines), so there is a working reference implementation in-tree.2. Event-log rows do not get entity resolution
Actor and subject columns come through as ids, blanks, or unmapped names, and each consumer papers over it differently.
Citizen #456767; allsubject/settlementblankbyPolluter: [],top_citizens: [],top_stations: [{"(unknown)": 6858.75}]ReputationTransfergiver column unrecognised after six candidate namesWorth a single shared resolver and one debug-level log of unmapped column keys, rather than four independent guesses at column naming.
3. No response-size policy was ever set
This issue. Not four defects — one design decision that has not been made.
limit/page/top_nanywhereget_currencyfilter addsselectedbut never narrowsget_progressionper-citizen timelines, 266 KB of 275 KB4. Derived and narrative fields computed loosely over correct raw data
The raw layer is sound (see below). These are all summary-layer arithmetic and wording.
byCitizensums quantities under an event-shaped name, exceedingtotalEvents15×settlementsFoundedadds foundings to foundation placementstotalCulture: 0republished as a KPI beside 910 culture of progressTail — genuine one-offs
#219 (
get_speciesown credential path), #228 (empty exception text), #229 (map seam wrap), #233 (explain_itemcategory ignored), #238 (get_government.scope), #239 (watcher label, cosmetic).What is not broken
Recording this so the issue count is not read as "the ingest is unsound". Every arithmetic check against the raw layer passed:
get_world— nineperActionCountssum exactly tototalEvents(13,333)get_crafting_atlas— four sum exactly tototalEvents(18,092)DigOrMine3,129 andChopTree1,331 agree across two independently-computed toolsget_civicsturnout: 90 / (90 + 218) = 0.2922 ✓get_trades—rollupTrades+detailedTrades=totalTrades✓find_trade,trade_watchers,get_stores,get_progression,get_worldall returned correct, useful dataNothing crashed, nothing was unsafe, and no data was lost. This is an accuracy-and-trust problem in the presentation layer, not a pipeline problem.
Suggested order
Tracking: 7 of 22 MCP tools exceed the client response cap, and no tool offers a limit or pagination parameterto Tracking: 2026-08-12 full-surface QA sweep of the eco MCP (22 tools, 23 findings)Scope note in the comment above is now resolved — this issue has been retitled and rewritten as the tracking issue for the whole sweep. The original payload-size content is preserved in the body under "Family 3 detail: response size", so the measurements referenced by #230 / #231 / #232 are unchanged and still here.
Ignore the "retitle or move if you'd rather keep this one narrow" line; that already happened.
Supersedes the two comments above. They used short-form
#NNNcross-references, which are ambiguous post-org-migration. Reposting the clustering with canonical URLs; the earlier two comments are retained for history only and should not be used for triage.Root-cause clustering for the 2026-08-12 sweep
23 findings, not 23 independent defects. Four families plus a tail of one-offs. Fix order below is by how much each family unblocks.
1. Currency is never parsed out of the trade ledger
get_tradesreturnscurrency: ""on all 526 detailed rows while settingtrade_currency_column_seen: true. Everything downstream that needs a currency name then fails in its own way.get_stores.currencies[]empty on stores with 4.6k volumefair_pricein-game side null (partly)get_marketempty (unconfirmed dependency, same data path)One fix, up to four issues. Start here.
find_traderesolves currency correctly from live store data (Spectres, Racines), so there is a working reference implementation in-tree.2. Event-log rows do not get entity resolution
Actor and subject columns come through as ids, blanks, or unmapped names, and each consumer papers over it differently.
subject/settlementblankbyPolluter: [],top_citizens: [],top_stations: [{"(unknown)": 6858.75}]ReputationTransfergiver column unrecognised after six candidate namesWorth a single shared resolver plus one debug-level log of unmapped column keys, rather than four independent guesses at column naming.
3. No response-size policy was ever set
One design decision that has not been made, not four defects. Measurements are in the issue body above.
get_currencyfilter addsselectedbut never narrowsget_progressionper-citizen timelines, 266 KB of 275 KB4. Derived and narrative fields computed loosely over correct raw data
The raw layer is sound. These are all summary-layer arithmetic and wording.
byCitizensums quantities under an event-shaped name, exceedingtotalEvents15×settlementsFoundedadds foundings to foundation placementstotalCulture: 0republished as a KPI beside 910 culture of progressTail — genuine one-offs
get_speciesown credential pathexplain_itemcategory ignoredget_government.scopeSuggested order
All 23 findings are closed. Summary of what landed, by the families this issue named.
Family 1 — currency never parsed out of the trade ledger
Root cause: the exporter keys a
CurrencyTraderow by the Currency object's id, whileCreateCurrencywrites its name, and nothing joined them. Both consumers mishandled the same column in opposite directions —currency.pyminted a phantom currency named after the id,trades.pyblanked it as a misalignment artifact.No export carried both halves of the key, so the stores/economy exporter mod now emits the id alongside the name and eco-app joins on it once, over the shared parsed rows, so the ledger, store directory and market boards all inherit it.
mixedCurrencyVolumeon a scalar that adds unlike unitsget_marketbuilds markets again, and explains itself when it cannotfair_pricein-game evidence, plus the silentIronIngot→ iron ore substitutionFamily 2 — event-log rows get no entity resolution
Citizen #<id>people. Two of the observed ones were election title ids. Blanksubject/settlementfields now resolve or surface assubjectId/settlementId.citizensGainedstill counts events and says so,distinctCitizensGainedcounts people, and exact repeats are dropped from the browsable list(unknown)station bucket is gone,has_datatells the truth, values are labelledemissionsrather thancount, and both tools name the columns they triedFamily 3 — no response-size policy
get_currency's filter narrows for real, a miss returns suggestions instead of the corpus, and the 185 repeated holder notes are hoisted to oneget_progressionis summary-first, withinclude_timelinesand acitizenargumentFamily 4 — derived and narrative fields computed loosely
growingstate instead of being called decliningcountsblock that reconciles thembyCitizenis an event count matchingget_world; iterations moved tobyCitizenIterationsobserved_per_daytotalCulture: 0reconciles against milestone progress, with the source namedTail
get_speciesuses the same admin credential path as every other toolexplain_itemechoes its category and explains empty factsget_government.scopedescribes the querytrade_watcherskeeps the supplied labelFiled along the way
world.animals: 0, split out of 237 because it has no in-payload reconciliation source and wants an upstream look firstState
ward exec testis green at 767 passed, from 631 passed / 1 failed when this started.ward exec lintandward exec smokepass; smoke runs against the live Sirens server and was used to confirm the culture and site-link fixes end to end.Closing this tracker: every finding it indexes is resolved.