Invariant: an unreadable dataset must report null, never zero — sweep all 25 MCP tools #266
Labels
No labels
burndown-2026-06
headless
autonomy
async-consult
autonomy
epic
autonomy
headless
autonomy
live-collab
c#
priority
P0
priority
P1
priority
P2
priority
P3
priority
P4
role/ai
role/creator
role/design
role/director
role/engineer
role/exec
role/human
role/ops
role/qa
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
coilyco-gaming/eco-app#266
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Why this is filed as an invariant, not a tool bug
Three separate commits have already fixed this exact shape, one tool at a time:
abcf1b0—fix(status): report absent /info numbers as null, never zero19513c1—fix(civics): report null for exporters that could not be readdd09786—fix(economy): separate unmeasured from measured-zero, and read the treasuryThe shape is still live in three tools at
5e05296, including the onedd09786named. Fixing the next demonstrated instance has not converged across three rounds. The rule needs stating once and applying to the whole surface.This class matters more than its severity suggests: none of these tools error. They return well-formed, confident, wrong numbers, which a downstream caller cannot detect without cross-checking a sibling tool.
The rule
nullwhen the dataset behind it could not be read.0MUST mean the server reported no activity.get_economy.datasets_unavailableis the reference shape;get_world's per-datasetHTTP 401warnings are the reference behaviour.Confirmed violations at
5e05296(observed 2026-08-13)1.
get_economy.govt_fundsreports 0 for a funded treasuryAgainst
eco.coilysiren.me:3001:get_economy.govt_funds_notestates the value is read from "the same dataset get_currency reports asmoney.governmentHoldings". Same dataset, same server, same minute, two answers.datasets_unavailable: []actively asserts nothing was missing.wages_total,taxes_paid,govt_funds_receivedandnet_tax_floware also 0, andhealth: "healthy"plus the narrative "Economy is healthy" are derived from them.2.
get_currencyconverts an auth failure into a claim about the worldAgainst
daokingdom.eu:3001:The server has 40 active currencies and 1,773 trades.
get_worldagainst the same host in the same session returnsHTTP 401for all nine datasets it tries, so the access failure is unambiguous and a sibling tool already reports it correctly.admin_ok: trueis also wrong here.3.
price_reciperanks the least-known recipe cheapestprice_recipe(product=IronBarItem)returns three recipes:RecycledIronBarIronBarSmeltIronSorted by cost, the recipe where nothing could be priced wins by 7×.
ingredientCost: 0here means "no leaf resolved", not "ingredients are free".perUnitCostcorrectly goesnullwhencomplete: false;totalCostandingredientCostdo not, so the only comparable-looking number is the corrupted one.Same tool against a server with no readable market (
daokingdom.eu:3001,product=SteelAxeItem) returnstotalCost: 0with seven entries inunpricedInputs.Credit where due:
unpricedInputsresolves leaves recursively and is correct. The bug is confined to the summed fields.Acceptance criteria
get_economy.govt_fundsreturns the treasury balance, ornullwith the dataset named indatasets_unavailable.get_currencyreports unreadable-ledger as unreadable;admin_okreflects actual admin reachability per server.price_recipe.totalCostand.ingredientCostarenullwhenevercomplete: false, matchingperUnitCost.nulland a populated unavailable-list — rather than one test per tool.Verification note
The three violations above were each confirmed by cross-checking a second tool on the same server in the same session, not inferred from reading code.
Claiming violation 3 only —
price_recipe. Angie (ENG,claudeseat), 20 minutes from this comment after the one minute buffer. Not claiming the sweep.I am being explicit about the boundary because this issue is deliberately framed as an invariant across 25 tools, and taking a slice of it without saying so would be the thing your own body warns about: "Fixing the next demonstrated instance has not converged across three rounds."
What I am taking
Sorted by cost, the recipe where nothing resolved wins by 7x.
ingredientCost: 0means "no leaf priced", not "ingredients are free", and it is the only comparable-looking number in the row.The correct behaviour is already in the same payload.
perUnitCostgoesnullwhencomplete: false.totalCostandingredientCostdo not. So this is one function agreeing with itself rather than a new rule.That makes it the cheapest of your three violations and the one least likely to need a design call.
Why not the other two now
get_economy.govt_fundsreturning 0 against a treasury of 87,912.99 needs the dataset wiring understood before I touch it, anddatasets_unavailable: []asserting nothing was missing is a second bug in the same tool. Bigger than it looks.get_currencyturning a 401 into "no currencies have been created or traded yet" spans auth handling and narrative generation, andadmin_ok: trueis wrong on the same payload.Both are real and both deserve more than the end of a session.
What I will not do
Claim this issue closed. Your acceptance is an audit of all 25 tools plus a generic invariant test, and one violation fixed is not that. I will reference this issue and leave it open.
I will also report if the fix makes an existing test red rather than adjusting the test —
price_recipe's ordering is presumably asserted somewhere, and if a test currently encodes the wrong ranking that is worth seeing rather than quietly rewriting.Same defect class as #269, which I fixed a few minutes ago: a number that looks measured and is not.
Violation 3 built. PR #272. Claim released. This issue stays open — it is a slice, not the sweep.
One payload now agrees with itself.
perUnitCostalready did this; the two summed fields did not.The frontend already knew the right answer
This is the part worth recording, because it shipped for free and I did not have to design it:
The recipe sort already treats an unknown cost as infinitely expensive, so an unpriced recipe sorts last rather than winning. That is exactly the behaviour your
IronBarItemtable wanted, and it was already correct forperUnitCost— it just never applied to the field the tool actually reported. Widening the two types tonumber | nullmakes the existing sort do the right thing for them too.The ingredient-cost cell takes the same null guard the per-unit cell already had.
What is not lost
ingredients[]still carries every resolved leaf andunpricedInputsstill names what did not, so a caller can see how far the rollup got. Your note thatunpricedInputsresolves recursively and is correct still holds — I did not touch it. Only the summary fields that looked comparable and were not.Validation
No existing test changed.
test_cost.py:248'singredientCost == 26.0assertion passes untouched, because that rollup is complete — which is the distinction the fix turns on.Mutation: restoring the partial sum fails
test_an_incomplete_rollup_reports_no_total_rather_than_zeroand nothing else.Still open here, and I am not claiming otherwise
get_economy.govt_fundsreturning 0 against a treasury of 87,912.99, withdatasets_unavailable: []asserting nothing was missing.get_currencyturning a 401 into "no currencies have been created or traded yet", withadmin_ok: trueon the same payload.Your framing is right that fixing one demonstrated instance at a time has not converged across three rounds. This is a fourth instance and I am not pretending it is the rule.
limitmust bound every unbounded array, and truncation must always warn — sweep all 25 MCP tools #267Claiming violation 1,
get_economy.govt_funds. Angie (ENG,claudeseat), 20 minutes from this comment after the one minute buffer.I deferred this earlier saying it needed the dataset wiring understood first. That was the right instinct and the wrong conclusion — understanding it is the work, and I have since done the same kind of tracing on four other tools in this repository.
Why this one ahead of anything left on #267
Everything I have fixed today was a bound or a caveat: a payload too big, a truncation unannounced, a check reporting success it never ran. This one returns a wrong number and builds a verdict on it.
Same dataset, same server, same minute, two answers. And
get_economy's owngovt_funds_notesays the value comes from "the same dataset get_currency reports asmoney.governmentHoldings", so the tool documents the contradiction it is producing.wages_total,taxes_paid,govt_funds_receivedandnet_tax_floware also 0, andhealth: "healthy"plus the narrative are derived from them — so a reader is not merely given a wrong figure, they are given a conclusion drawn from it.datasets_unavailable: []is the part that makes it undetectable: the tool actively asserts nothing was missing.What I will establish before changing anything
Whether the treasury read fails or returns zero. Those need opposite fixes — a failed read must become
nullwith the dataset named, per your rule 1 and 4; a genuine zero must stay0, per rule 2. Getting that backwards would replace a wrong number with a wrong null, anddd09786already did this work once for other fields in this tool, so there is a pattern to follow rather than invent.Whether
healthand the narrative can be computed at all when an input is null. Your rule 3 says a narrative must not assert a fact resting on an unread dataset. If the answer is thathealthbecomes unavailable rather than "healthy", that is a behaviour change worth naming rather than sliding in.If it turns out the read succeeds and the value is genuinely lost in translation, I will say so — that is a different bug from the one this issue describes.
Not taking
get_currency's 401-as-a-claim andadmin_ok: true, andprice_recipewhich #272 already fixed. The 25-tool audit and generic invariant test stay open too.Claiming violation 2,
get_currency. Angie (ENG,claudeseat), 20 minutes from this comment after the one minute buffer.#279 fixed violation 1 by finding that
_fetch_datasetreturned an empty list for any 200, so an unparseable body became a confident zero. This is the same family and the last violation on your list.What makes it the sharpest of the three
The payload contradicts itself:
A reader does not need a second tool to catch this one. Violation 1 needed
get_currencyto disproveget_economy; here the disproof is three lines down in the same response.And
get_worldagainst the same host returnsHTTP 401for all nine datasets it tries, so the access failure is unambiguous and a sibling tool already reports it correctly — which is your rule 4's reference behaviour.What I will establish first
Whether
admin_ok: trueis wrong or is answering a different question. It may mean "a token was configured" rather than "the admin surface answered", in which case it is technically true and badly named, and the fix is different from making it false.fetch_currencysets it near the token load, which is what makes me suspect the former.Where the narrative is generated, and whether it can see the failure. Your rule 3 says a narrative must not assert a fact resting on an unread dataset. If the narrative is built from
counts.totalalone it cannot know, and the fix belongs where that count is derived rather than in the wording.What I am not doing
Touching the holder cap. That is #267's rule 3, it sits behind a cached fetch, and I documented the trap there rather than walking into it. Same tool, different defect, and I am not bundling them.
I will report if the narrative turns out to be correct for a genuinely empty server and only wrong under 401 — that would make this a smaller fix than the issue implies, and worth saying.
warningsis the last key in the response, so a truncating consumer destroys the caveats first and the rows arrive looking complete #304headlessexist in this repo, one repo-scoped and one org-scoped, so the dispatch gate's vocabulary is ambiguous here #305