QA sweep 2026-08-13: remaining verified findings across the 25-tool MCP surface #270

New issue

Open

opened 2026-08-13 19:48:38 +00:00 by coilyco-ops · 0 comments

coilyco-ops commented

2026-08-13 19:48:38 +00:00

Member

🤖 Filed by Claude Code on Kai's behalf.

Collector for findings from an independent QA pass over all 25 Eco MCP tools at 5e05296, run 2026-08-13 18:21–18:30 UTC — sixteen minutes after that commit was pushed. Criticals are filed separately as #266, #267, #268, #269; this issue holds everything else so it does not get lost.

Coverage caveat, stated plainly: 71 calls across 25 tools. Eight tools reached five calls; seventeen did not. Several tools return 30–60 KB per call even with limit set, so exhaustive per-tool repetition did not fit one context. Breadth was chosen over depth. That payload size is #267.

No regressions found. Everything I retested from the 2026-08-12 and 2026-08-13 fix waves still works: get_civics id handling, get_map seam flags, find_trade item matching, get_stores mixed-currency flagging, get_recipes indexScope, get_social reputation columns, get_market rebuild, get_milestones culture reconciliation.

Correctness

get_climate headline contradicts its own explainer. status: "warming" and narrative "Climate is warming" alongside an explainer reading "CO2 sits at 325 ppm — pinned to the simulation floor, the cleanest the atmosphere can get" and "the current trajectory is safe", with temperature.rate_per_day: 0. A caller reading status and a caller reading explainer reach opposite conclusions.
get_region labels a growing population "declining". WheatSpecies has changePct: +0.454 (500 → 727 across the cycle) with state: "declining" and reason "Population is declining, but the combined at-risk threshold was not crossed." The recent window is down 17%, which is what the classifier keyed on — but stable entries carry the qualifier "…but the population is still well down across the cycle", and the mirror-image qualifier is missing here. 32db802 made the classifier signed; the reason string did not follow.
get_region.sharePercent silently excludes water. CoastalWater is the single largest biome at percent: 36 but reports sharePercent: 0; shares are computed over the 39% land subtotal only. A caller reading sharePercent concludes Ocean (33.3%) leads and CoastalWater is absent. percent, rawSumPercent, classifiedPercent and unclassifiedPercent all reconcile correctly — only sharePercent needs a denominator note or a rename.
get_progression timelines are undeduplicated and non-monotonic. One citizen's characterLevel: 10 with a timeline descending 8, 8, 7, 7, 6, 6, 5, 5, 4, 4 in reverse-chronological order, then level 10 at day 17 and level 8 at day 14. Every level appears twice. get_civics explicitly dedupes this same exporter's repeated rows (132e0c2, 2357c0b) and documents it; get_progression does not.
get_species fabricates identity for unknown input. get_species(name="NotASpeciesZZZ") returns name: "Not A Species Z Z Z" (input mangled by splitting on capitals), speciesId: "NotASpeciesZZZSpecies" (an id invented for a species that does not exist), and error: "population unavailable (HTTP 500)" — a raw upstream 500, with no not_found flag. explain_item handles the same case correctly with not_found: true; copy that shape.
get_species taxonomy rank/name mismatch. Bison returns {"rank": "Genus", "name": "Cattle and bison"}. That is the iNaturalist vernacular for tribe Bovini; the genus is Bison.
find_trade counts unbuyable offers and mislabels a supply gap. Offers with quantity: 0 appear in both cheapest and resale. One supplyGaps entry is labelled reason: "thin_supply" with demandQty: 0 and supplyQty: 147 — that is excess supply, not thin supply. Separately arbitrage: [] is returned with no explanation while Iron Bar sells at 0.26 in one currency and bids at 1.00 in another; if cross-currency arbitrage is deliberately excluded, say so in the payload.
trade_watchers compares prices across unlike currencies. An "under 0.85" threshold is applied to trades spanning two currencies, and totalVolume sums them. get_stores flags exactly this with mixedCurrencyVolume (48ca4d7) and find_trade segments by currency; trade_watchers does neither.

Naming and disclosure

get_server_status.activeAndOnline is not "online". Default server: online: 0, onlineNames: [], activeAndOnline: 7. Cross-check on daokingdom.eu:3001: online: 27 with 27 names, activeAndOnline: 62. The field is Eco's ActivePlayers; the name asserts a conjunction the data contradicts.
get_server_status.animalsNote is factually stale. It reads "Upstream /info reports 0 animals on every server observed, including ones with live fauna". daokingdom.eu:3001 returned animals: 773 in the same batch. The note from f7a12a2 is right to exist; its universal claim is now false.
explain_item.image_credit duplicates image_url — no author, no licence, which is what actually matters for Wikimedia reuse. get_species.photoAttribution does this correctly ("(c) CIAT, some rights reserved (CC BY-SA)"), so the right shape already exists in-suite.
explain_item category facts rarely populate. Quartz with category=mineral returns "Wikidata returned no mineral facts for 'Quartz' (looked for: Mohs hardness)"; the Wikidata entity for quartz carries Mohs hardness 7. Only 1 of 5 calls returned any fact. 9fbdb4e claims to resolve entity facts to labels; the resolution still misses.
get_milestones reports eight different "current" culture values (27.71 … 910.32) for one world total, because Eco freezes each figure at completion. d504868 reconciled the 0-vs-910 discrepancy well; this second discrepancy has no note.
get_civics cannot identify what was elected. recentElections, recentOutcomes and recentSettlements return subject: null with only subjectId, and proposerId: null while proposer resolves to a name.

Presentation

Eco colour markup leaks into names and keys. find_trade store labels and — more seriously — storeKey values embed <color=red>…</color>. get_map deed names likewise. A key carrying markup is load-bearing and fragile.
Float noise surfaced raw. Prices as 0.100000000692029 and 0.9100000063749999, costs as 22.483333352864243, timeCost: 0.020000000000000004. Currency values want rounding at the presentation edge.
get_government law previews are mangled. Raw \r\n survives, and preview_lines contains "Wet Tailingsif Dropped" (missing separator) and "16 Tools16 Tools" (duplicated run).
get_recipes station filter rejects its own documented example. The schema says "Accepts the id or the display name, e.g. AnvilItem or 'Anvil'". station="Anvil" returns "no station named 'Anvil' exists in this recipe graph… Did you mean: AnvilObject?" Display-name resolution works for product and skill but not station, and the id in the doc (AnvilItem) is also wrong — it is AnvilObject. Either fix the resolver or fix the docstring.
get_currency contradicts itself in every payload. holders_reachable: true sits beside holders_unavailable_note: "…it is not reachable right now", while individual currencies carry holders.reachable: false.
get_currency not-found suggestions are unactionable. A miss returns suggestions containing raw numeric currency ids ("2533707", "2967954"), not names.
get_social reputation graph is dominated by one unflagged actor. topReputationGivers leads with 2,788 against 115 for the next — round-number grants (204 / 136 / 68) to 26 distinct targets, consistent with a system or admin actor. It shapes the "Most-repped" headline with no outlier flag.
get_market omits the reconciliation note its siblings carry. get_trades.counts.note and get_economy.trades_total_note both explain that 3,680 exporter rows = 528 detail + 3,152 rollups covering 22,405 merged = 22,933 events, and that /info counts a different population again. get_market reports totalTrades: 22933 with no such note, and its empty-filter warning cites a fourth number ("528 ledger rows").

Verified correct — do not regress

Arithmetic and cross-tool consistency I checked independently, all of which passed:

Cost engine: CookingUtensils = 2 × IronBar @ 0.90 (market) + 50 cal × 0.001 + 0.4 min × 0.05 = 1.87 exactly; the 0.90 matches get_market's IronBar median.
SteelAxe: 20 × 0.7125 + 8 × 0.80 + 5 × 0.366667 = 22.4833 exactly.
trade_watchers IronBar totalVolume 548.520016671 == get_trades.byItem IronBarItem 548.520016671 (exact).
Wheat: get_market volume 638 @ 0.10 == get_trades.byItem 63.8 currency.
get_species Bison populationLatest 118 == get_region BisonSpecies.current 118.
get_government.active_laws_count 5 == /info laws 5.
get_map.worldDim 1000×1000 == /info "1km²"; deedCount 67 + seamCopyCount 1 == polygonCount 68.
get_civics turnout 90 / (90 + 218) = 0.2922; netDistinctCitizens 69 − 16 = 53.
get_climate (427.4 − 400) / 25 = 1.1 == peak_drives_c and peak_drives_m.
get_region.speciesRisk counts: 9 at_risk + 1 declining + 1 recovering + 3 growing + 80 stable = 94 == speciesSeen.
trade_watchers full lifecycle: fuzzy match ("Iron Bar" → IronBarItem), threshold filtering correct (1 of 5 trades under 0.85), advance=false honoured, create-validation rejects a price watcher with no operator, removal works, bogus id returns removed: false rather than erroring.

Fixture hygiene

trade_watchers was the only write path exercised. The store was empty beforehand; two watchers were created, evaluated with advance=false so no watcher state advanced, then removed, and action=list confirmed empty afterwards. No residue.

> 🤖 Filed by Claude Code on Kai's behalf. Collector for findings from an independent QA pass over all 25 Eco MCP tools at `5e05296`, run 2026-08-13 18:21–18:30 UTC — sixteen minutes after that commit was pushed. Criticals are filed separately as #266, #267, #268, #269; this issue holds everything else so it does not get lost. **Coverage caveat, stated plainly:** 71 calls across 25 tools. Eight tools reached five calls; seventeen did not. Several tools return 30–60 KB per call even with `limit` set, so exhaustive per-tool repetition did not fit one context. Breadth was chosen over depth. That payload size is #267. **No regressions found.** Everything I retested from the 2026-08-12 and 2026-08-13 fix waves still works: `get_civics` id handling, `get_map` seam flags, `find_trade` item matching, `get_stores` mixed-currency flagging, `get_recipes` `indexScope`, `get_social` reputation columns, `get_market` rebuild, `get_milestones` culture reconciliation. --- ## Correctness - **`get_climate` headline contradicts its own explainer.** `status: "warming"` and narrative "Climate is warming" alongside an explainer reading "CO2 sits at 325 ppm — pinned to the simulation floor, the cleanest the atmosphere can get" and "the current trajectory is safe", with `temperature.rate_per_day: 0`. A caller reading `status` and a caller reading `explainer` reach opposite conclusions. - **`get_region` labels a growing population "declining".** `WheatSpecies` has `changePct: +0.454` (500 → 727 across the cycle) with `state: "declining"` and reason "Population is declining, but the combined at-risk threshold was not crossed." The recent window is down 17%, which is what the classifier keyed on — but `stable` entries carry the qualifier "…but the population is still well down across the cycle", and the mirror-image qualifier is missing here. `32db802` made the classifier signed; the reason string did not follow. - **`get_region.sharePercent` silently excludes water.** `CoastalWater` is the single largest biome at `percent: 36` but reports `sharePercent: 0`; shares are computed over the 39% land subtotal only. A caller reading `sharePercent` concludes Ocean (33.3%) leads and CoastalWater is absent. `percent`, `rawSumPercent`, `classifiedPercent` and `unclassifiedPercent` all reconcile correctly — only `sharePercent` needs a denominator note or a rename. - **`get_progression` timelines are undeduplicated and non-monotonic.** One citizen's `characterLevel: 10` with a timeline descending 8, 8, 7, 7, 6, 6, 5, 5, 4, 4 in reverse-chronological order, then level 10 at day 17 and level 8 at day 14. Every level appears twice. `get_civics` explicitly dedupes this same exporter's repeated rows (`132e0c2`, `2357c0b`) and documents it; `get_progression` does not. - **`get_species` fabricates identity for unknown input.** `get_species(name="NotASpeciesZZZ")` returns `name: "Not A Species Z Z Z"` (input mangled by splitting on capitals), `speciesId: "NotASpeciesZZZSpecies"` (an id invented for a species that does not exist), and `error: "population unavailable (HTTP 500)"` — a raw upstream 500, with no `not_found` flag. `explain_item` handles the same case correctly with `not_found: true`; copy that shape. - **`get_species` taxonomy rank/name mismatch.** `Bison` returns `{"rank": "Genus", "name": "Cattle and bison"}`. That is the iNaturalist vernacular for tribe Bovini; the genus is *Bison*. - **`find_trade` counts unbuyable offers and mislabels a supply gap.** Offers with `quantity: 0` appear in both `cheapest` and `resale`. One `supplyGaps` entry is labelled `reason: "thin_supply"` with `demandQty: 0` and `supplyQty: 147` — that is excess supply, not thin supply. Separately `arbitrage: []` is returned with no explanation while Iron Bar sells at 0.26 in one currency and bids at 1.00 in another; if cross-currency arbitrage is deliberately excluded, say so in the payload. - **`trade_watchers` compares prices across unlike currencies.** An "under 0.85" threshold is applied to trades spanning two currencies, and `totalVolume` sums them. `get_stores` flags exactly this with `mixedCurrencyVolume` (`48ca4d7`) and `find_trade` segments by currency; `trade_watchers` does neither. ## Naming and disclosure - **`get_server_status.activeAndOnline` is not "online".** Default server: `online: 0`, `onlineNames: []`, `activeAndOnline: 7`. Cross-check on `daokingdom.eu:3001`: `online: 27` with 27 names, `activeAndOnline: 62`. The field is Eco's ActivePlayers; the name asserts a conjunction the data contradicts. - **`get_server_status.animalsNote` is factually stale.** It reads "Upstream /info reports 0 animals on every server observed, including ones with live fauna". `daokingdom.eu:3001` returned `animals: 773` in the same batch. The note from `f7a12a2` is right to exist; its universal claim is now false. - **`explain_item.image_credit` duplicates `image_url`** — no author, no licence, which is what actually matters for Wikimedia reuse. `get_species.photoAttribution` does this correctly ("(c) CIAT, some rights reserved (CC BY-SA)"), so the right shape already exists in-suite. - **`explain_item` category facts rarely populate.** `Quartz` with `category=mineral` returns "Wikidata returned no mineral facts for 'Quartz' (looked for: Mohs hardness)"; the Wikidata entity for quartz carries Mohs hardness 7. Only 1 of 5 calls returned any fact. `9fbdb4e` claims to resolve entity facts to labels; the resolution still misses. - **`get_milestones` reports eight different "current" culture values** (27.71 … 910.32) for one world total, because Eco freezes each figure at completion. `d504868` reconciled the 0-vs-910 discrepancy well; this second discrepancy has no note. - **`get_civics` cannot identify what was elected.** `recentElections`, `recentOutcomes` and `recentSettlements` return `subject: null` with only `subjectId`, and `proposerId: null` while `proposer` resolves to a name. ## Presentation - **Eco colour markup leaks into names and keys.** `find_trade` store labels and — more seriously — `storeKey` values embed `<color=red>…</color>`. `get_map` deed names likewise. A key carrying markup is load-bearing and fragile. - **Float noise surfaced raw.** Prices as `0.100000000692029` and `0.9100000063749999`, costs as `22.483333352864243`, `timeCost: 0.020000000000000004`. Currency values want rounding at the presentation edge. - **`get_government` law previews are mangled.** Raw `\r\n` survives, and `preview_lines` contains `"Wet Tailingsif Dropped"` (missing separator) and `"16 Tools16 Tools"` (duplicated run). - **`get_recipes` station filter rejects its own documented example.** The schema says "Accepts the id or the display name, e.g. AnvilItem or 'Anvil'". `station="Anvil"` returns "no station named 'Anvil' exists in this recipe graph… Did you mean: AnvilObject?" Display-name resolution works for `product` and `skill` but not `station`, and the id in the doc (`AnvilItem`) is also wrong — it is `AnvilObject`. Either fix the resolver or fix the docstring. - **`get_currency` contradicts itself in every payload.** `holders_reachable: true` sits beside `holders_unavailable_note: "…it is not reachable right now"`, while individual currencies carry `holders.reachable: false`. - **`get_currency` not-found suggestions are unactionable.** A miss returns `suggestions` containing raw numeric currency ids (`"2533707"`, `"2967954"`), not names. - **`get_social` reputation graph is dominated by one unflagged actor.** `topReputationGivers` leads with 2,788 against 115 for the next — round-number grants (204 / 136 / 68) to 26 distinct targets, consistent with a system or admin actor. It shapes the "Most-repped" headline with no outlier flag. - **`get_market` omits the reconciliation note its siblings carry.** `get_trades.counts.note` and `get_economy.trades_total_note` both explain that 3,680 exporter rows = 528 detail + 3,152 rollups covering 22,405 merged = 22,933 events, and that `/info` counts a different population again. `get_market` reports `totalTrades: 22933` with no such note, and its empty-filter warning cites a fourth number ("528 ledger rows"). ## Verified correct — do not regress Arithmetic and cross-tool consistency I checked independently, all of which passed: - Cost engine: `CookingUtensils` = 2 × IronBar @ 0.90 (market) + 50 cal × 0.001 + 0.4 min × 0.05 = **1.87** exactly; the 0.90 matches `get_market`'s IronBar median. - `SteelAxe`: 20 × 0.7125 + 8 × 0.80 + 5 × 0.366667 = **22.4833** exactly. - `trade_watchers` IronBar `totalVolume` 548.520016671 == `get_trades.byItem` IronBarItem 548.520016671 (exact). - Wheat: `get_market` volume 638 @ 0.10 == `get_trades.byItem` 63.8 currency. - `get_species` Bison `populationLatest` 118 == `get_region` `BisonSpecies.current` 118. - `get_government.active_laws_count` 5 == `/info` `laws` 5. - `get_map.worldDim` 1000×1000 == `/info` "1km²"; `deedCount` 67 + `seamCopyCount` 1 == `polygonCount` 68. - `get_civics` turnout 90 / (90 + 218) = 0.2922; `netDistinctCitizens` 69 − 16 = 53. - `get_climate` (427.4 − 400) / 25 = 1.1 == `peak_drives_c` and `peak_drives_m`. - `get_region.speciesRisk` counts: 9 at_risk + 1 declining + 1 recovering + 3 growing + 80 stable = 94 == `speciesSeen`. - `trade_watchers` full lifecycle: fuzzy match ("Iron Bar" → `IronBarItem`), threshold filtering correct (1 of 5 trades under 0.85), `advance=false` honoured, create-validation rejects a price watcher with no operator, removal works, bogus id returns `removed: false` rather than erroring. ## Fixture hygiene `trade_watchers` was the only write path exercised. The store was empty beforehand; two watchers were created, evaluated with `advance=false` so no watcher state advanced, then removed, and `action=list` confirmed empty afterwards. No residue.

coilyco-ops added the

labels

2026-08-15 21:09:02 +00:00