The model is shown the reachable host list and fetches unreachable hosts anyway, which is a different bug from the one #940 asked to fix #1135

New issue

Open

opened 2026-08-23 01:24:00 +00:00 by coilyco-ops · 0 comments

coilyco-ops commented

2026-08-23 01:24:00 +00:00

Member

Filed by Darren (director seat), 2026-08-23. #940 closed at 01:23:01 with #1132 carrying its oversized-results half. Its other half was not carried, and it is the more interesting one.

What 940 originally asked for, and why that ask was void

It asked for either of two levers, and called the first much cheaper:

Tell the model the allowlist. Render the reachable host set into the fetch tool's description so a refusal is predictable rather than discovered.

That lever was already built when the issue was filed. internal/community/fetch.go:65 composes the fetch tool's description as "Fetch a page over HTTPS and return its text. Only these hosts are reachable: " + strings.Join(s.hosts, ", ") + ". Nothing else is.", and fetch_test.go:48 fails if it stops naming them. It landed in 4795f1c on 2026-08-13, five days before 940 was written. I filed 940 and I missed it.

So the real finding is the one nobody asked about

The model is shown the reachable host list and fetches unreachable hosts anyway.

The measurement from 940 stands unchanged. In trace b10fb59b0a5e3d00245bca11f706fef0, six fetch_url calls in one turn returned 56, 130, 130, 130, 24, and 24 bytes. Those are refusals rather than content. The turn spent 182.46 seconds and six tool rounds to produce 1,246 bytes, with rounds three and four going almost entirely to those failing fetches and the searches that fed them.

That is not a missing-instruction problem. It is an instruction-not-used problem, and those have completely different fixes.

Why it is newly measurable

#1120 changed fetch_url so a non-text response returns its type and length rather than decoded garbage. A refusal and a real asset are now distinguishable to the caller, which they were not when 940 was written, so the failure can be counted rather than inferred from byte sizes.

Candidate explanations, none tested

The list is too long to use. SIRENS_ECHO_FETCH_HOSTS on the dowel lane is 28 hosts plus wildcards, rendered as one comma-joined sentence inside a tool description.
It arrives too far from the call. The description is read at tool-selection time, many rounds before the model composes a URL from a search result.
It is ignored. The model has just run create_web_search, which returns the open web, and the pull toward fetching what it found may simply beat a constraint stated once.

Done when

The rate of refused fetch_url calls is measured over a real window, the leading explanation is named with evidence rather than assumed, and the chosen remedy is stated. The remedy may not be in this repository: filtering at the search boundary is the exa server's, and shortening or repositioning the host list is a prompt or roster decision.

#940 - closed, where this was measured and mis-scoped
#1120 - what makes a refusal countable
#1132 - the other half of 940

**Filed by Darren (director seat), 2026-08-23.** https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/940 closed at `01:23:01` with https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/1132 carrying its oversized-results half. **Its other half was not carried**, and it is the more interesting one. ## What 940 originally asked for, and why that ask was void It asked for either of two levers, and called the first much cheaper: > Tell the model the allowlist. Render the reachable host set into the fetch tool's description so a refusal is predictable rather than discovered. **That lever was already built when the issue was filed.** `internal/community/fetch.go:65` composes the fetch tool's description as `"Fetch a page over HTTPS and return its text. Only these hosts are reachable: " + strings.Join(s.hosts, ", ") + ". Nothing else is."`, and `fetch_test.go:48` fails if it stops naming them. It landed in `4795f1c` on 2026-08-13, five days before 940 was written. I filed 940 and I missed it. ## So the real finding is the one nobody asked about **The model is shown the reachable host list and fetches unreachable hosts anyway.** The measurement from 940 stands unchanged. In trace `b10fb59b0a5e3d00245bca11f706fef0`, six `fetch_url` calls in one turn returned 56, 130, 130, 130, 24, and 24 bytes. Those are refusals rather than content. The turn spent **182.46 seconds and six tool rounds to produce 1,246 bytes**, with rounds three and four going almost entirely to those failing fetches and the searches that fed them. That is not a missing-instruction problem. It is an instruction-not-used problem, and those have completely different fixes. ## Why it is newly measurable https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/pulls/1120 changed `fetch_url` so a non-text response returns its type and length rather than decoded garbage. **A refusal and a real asset are now distinguishable to the caller**, which they were not when 940 was written, so the failure can be counted rather than inferred from byte sizes. ## Candidate explanations, none tested * **The list is too long to use.** `SIRENS_ECHO_FETCH_HOSTS` on the dowel lane is 28 hosts plus wildcards, rendered as one comma-joined sentence inside a tool description. * **It arrives too far from the call.** The description is read at tool-selection time, many rounds before the model composes a URL from a search result. * **It is ignored.** The model has just run `create_web_search`, which returns the open web, and the pull toward fetching what it found may simply beat a constraint stated once. ## Done when The rate of refused `fetch_url` calls is measured over a real window, the leading explanation is named with evidence rather than assumed, and the chosen remedy is stated. **The remedy may not be in this repository**: filtering at the search boundary is the `exa` server's, and shortening or repositioning the host list is a prompt or roster decision. ## Related * https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/940 - closed, where this was measured and mis-scoped * https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/pulls/1120 - what makes a refusal countable * https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/1132 - the other half of 940

coilyco-ops added the

labels

2026-08-23 01:24:00 +00:00

coilyco-ops referenced this issue

2026-08-23 01:24:12 +00:00

Tool results are 87% discarded after the fact, and fetch_url burns rounds on hosts the allowlist was always going to refuse #940

coilyco-ops referenced this issue

2026-08-23 01:40:47 +00:00