Add a calculator tool: every number Echo computes herself is predicted, not calculated #916

New issue

Closed

opened 2026-08-17 18:10:18 +00:00 by coilyco-ops · 1 comment

coilyco-ops commented

2026-08-17 18:10:18 +00:00

Member

Filed by Darren (director seat) during a capability audit mapping Go stdlib surface areas against what Echo can actually reach. Kai chose this over the cheaper alternative of merely labelling unverified numbers as estimates.

The gap

There is no arithmetic anywhere in Echo's executable surface. Not a tool, not an in-process provider, not a roster server. Every number she produces that did not arrive inside a tool result is a language model predicting plausible digits.

That is fine for "roughly a third" and not fine for a total, a rate, a unit price, or a multiplication across a recipe tree.

Why it matters here specifically

Echo answers questions about an economy. Members ask what something costs, whether a trade is worth taking, what a recipe consumes at scale. Those are the questions where a confidently wrong number is indistinguishable from a right one, and #449 already documents the general shape of that failure: a bounded tool result becoming an unbounded claim, with no error and nothing for a gate to catch.

What already covers part of it

The eco-game MCP computes server-side and returns results rather than inputs:

fair_price
price_recipe
get_market, get_trades, get_economy

Where a member's question maps onto one of those, the number is real. The gap is everything else: converting units, summing a list the tools returned separately, scaling a recipe, comparing two prices from two calls, and any arithmetic a member simply asks for directly.

The choice, and what it forecloses

An honesty rule was the alternative, requiring any number not sourced from a tool to be marked as estimated. It fits existing grounding doctrine under #846 and needs no new capability. Not chosen. It makes Echo accurate about her own unreliability rather than reliable, and in a community that asks price questions the second is worth paying for.

The honesty rule is still worth having as the fallback for arithmetic the tool cannot express, so this issue does not close #846's territory.

Shape

In-process, alongside FetchProvider, ScratchProvider and RepoInventoryProvider in agent.go, rather than another MCP server. It needs no network, no auth, and no egress bound, and the roster is already 11 servers with two of them down per #897. An in-process provider is also the cheaper option against #859, which measures the skillpack and tool descriptions at 79% of Echo's prompt.

Deliberately not a general expression evaluator with arbitrary code execution. Arithmetic, units, and percentages. #179 puts code execution across the blast-radius line, and a calculator has no business approaching it.

Done means

A tool that evaluates arithmetic and returns the result, with the expression echoed back so the reasoning is auditable in telemetry.
Bounded input length and a refusal path, matching how fetch_url and the scratch tools already handle bad arguments.
A prompt rule directing that any arithmetic in a member-facing answer goes through it.

#449 - a bounded result becoming an unbounded claim. Same family of failure.
#846 - the measured-breach epic that owns grounding rules.
#859 - why this should be in-process rather than a twelfth server.

Filed by Darren (director seat) during a capability audit mapping Go stdlib surface areas against what Echo can actually reach. **Kai chose this over the cheaper alternative** of merely labelling unverified numbers as estimates. ## The gap There is no arithmetic anywhere in Echo's executable surface. Not a tool, not an in-process provider, not a roster server. Every number she produces that did not arrive inside a tool result is a language model predicting plausible digits. That is fine for "roughly a third" and not fine for a total, a rate, a unit price, or a multiplication across a recipe tree. ## Why it matters here specifically Echo answers questions about an economy. Members ask what something costs, whether a trade is worth taking, what a recipe consumes at scale. Those are the questions where a confidently wrong number is indistinguishable from a right one, and #449 already documents the general shape of that failure: a bounded tool result becoming an unbounded claim, with no error and nothing for a gate to catch. ## What already covers part of it The eco-game MCP computes server-side and returns results rather than inputs: * `fair_price` * `price_recipe` * `get_market`, `get_trades`, `get_economy` Where a member's question maps onto one of those, the number is real. The gap is everything else: converting units, summing a list the tools returned separately, scaling a recipe, comparing two prices from two calls, and any arithmetic a member simply asks for directly. ## The choice, and what it forecloses An **honesty rule** was the alternative, requiring any number not sourced from a tool to be marked as estimated. It fits existing grounding doctrine under #846 and needs no new capability. Not chosen. It makes Echo accurate about her own unreliability rather than reliable, and in a community that asks price questions the second is worth paying for. The honesty rule is still worth having as the fallback for arithmetic the tool cannot express, so this issue does not close #846's territory. ## Shape In-process, alongside `FetchProvider`, `ScratchProvider` and `RepoInventoryProvider` in `agent.go`, rather than another MCP server. It needs no network, no auth, and no egress bound, and the roster is already 11 servers with two of them down per #897. An in-process provider is also the cheaper option against #859, which measures the skillpack and tool descriptions at 79% of Echo's prompt. Deliberately not a general expression evaluator with arbitrary code execution. Arithmetic, units, and percentages. #179 puts code execution across the blast-radius line, and a calculator has no business approaching it. ## Done means * A tool that evaluates arithmetic and returns the result, with the expression echoed back so the reasoning is auditable in telemetry. * Bounded input length and a refusal path, matching how `fetch_url` and the scratch tools already handle bad arguments. * A prompt rule directing that any arithmetic in a member-facing answer goes through it. ## Related * #449 - a bounded result becoming an unbounded claim. Same family of failure. * #846 - the measured-breach epic that owns grounding rules. * #859 - why this should be in-process rather than a twelfth server.

coilyco-ops added the

labels

2026-08-17 18:10:18 +00:00

coilyco-ops referenced this issue

2026-08-17 18:26:01 +00:00

Skills are skill-shaped directories, not skills: every reference file is inlined into every prompt #859

coilyco-ops referenced this issue from a commit

2026-08-17 19:36:20 +00:00

feat: add a calculator tool, so a number is calculated rather than predicted

coilyco-ops referenced this issue from a pull request that will close it,

2026-08-17 19:36:40 +00:00

feat: add a calculator tool, so a number is calculated rather than predicted #926

coilyco-ops commented

2026-08-17 19:36:59 +00:00

Author

Member

Built and in review at #926. Angie (ENG, claude seat).

All three of your "done means" are in, and in process beside FetchProvider and ScratchProvider as you scoped it.

Exact rather than floating point, which the issue does not ask for and I think it wants. Values are big.Rat, so 0.1 + 0.2 is 0.3 and a recipe scaled across three multiplications does not drift. A result that is not a decimal says it was rounded and prints the exact fraction beside it, because a rounded number presented as exact is the same failure one step quieter.

Not a language. No identifiers and no calls in the grammar, so total * 2, exec(1), and rm -rf / all refuse with "that is not arithmetic". #179 puts code execution across the blast-radius line and this does not approach it.

Registered unconditionally. It needs no configuration, and a tool behind an unset switch is a tool nobody has, which is what #885 found after SIRENS_ECHO_DISCORD_COMMANDS sat dark.

Three things a reviewer should look at

Both prompt budgets rise 203 bytes for the rule routing arithmetic through the tool, recorded in docs/sirens-echo-prompt.md with its cause. The rule is what makes the tool used rather than merely offered, so the raise is the point. Worth knowing against #859, which I am picking up next.
Deep's board citations moved. The new prompt section shifted line numbers and boardcitation_test.go caught it, which is that test doing exactly its job.
Units are not in. You list them beside arithmetic and percentages. They need a conversion table rather than a parser, none of Echo's questions are physical-unit questions, and a half-built table is worse than none. I left it out rather than guessing at scope. Say the word and it is a separate change.

Built and in review at https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/pulls/926. Angie (ENG, `claude` seat). All three of your "done means" are in, and in process beside `FetchProvider` and `ScratchProvider` as you scoped it. **Exact rather than floating point**, which the issue does not ask for and I think it wants. Values are `big.Rat`, so `0.1 + 0.2` is `0.3` and a recipe scaled across three multiplications does not drift. **A result that is not a decimal says it was rounded** and prints the exact fraction beside it, because a rounded number presented as exact is the same failure one step quieter. **Not a language.** No identifiers and no calls in the grammar, so `total * 2`, `exec(1)`, and `rm -rf /` all refuse with "that is not arithmetic". #179 puts code execution across the blast-radius line and this does not approach it. **Registered unconditionally.** It needs no configuration, and a tool behind an unset switch is a tool nobody has, which is what #885 found after `SIRENS_ECHO_DISCORD_COMMANDS` sat dark. ## Three things a reviewer should look at * **Both prompt budgets rise 203 bytes** for the rule routing arithmetic through the tool, recorded in `docs/sirens-echo-prompt.md` with its cause. The rule is what makes the tool used rather than merely offered, so the raise is the point. Worth knowing against #859, which I am picking up next. * **Deep's board citations moved.** The new prompt section shifted line numbers and `boardcitation_test.go` caught it, which is that test doing exactly its job. * **Units are not in.** You list them beside arithmetic and percentages. They need a conversion table rather than a parser, none of Echo's questions are physical-unit questions, and a half-built table is worse than none. I left it out rather than guessing at scope. Say the word and it is a separate change.