feat(fetch): a read-only fetch, bounded by an allowlist

coilyco-ops commented

2026-08-13 12:38:40 +00:00

Member

closes #426

#412. Plain net/http, as Kai guessed.

Merge #424 first — main is currently red for an unrelated reason and this PR's green is against a fixed base.

The allowlist is the feature; the fetching is the easy part

This runs inside the cluster. Unbounded it reaches the tailnet, other services' internal endpoints, and cloud metadata addresses — and the model is exactly the component an attacker gets to talk to. A tool that fetches any URL a model can be persuaded to fetch is SSRF with a conversational interface.

Empty allowlist offers no tool at all: no schema, no prompt mention, nothing to be talked into.

Five bounds, and the one that is easy to get wrong

Exact host match, not suffix — registering eco-app.coilysiren.me.evil.example costs an attacker nothing
HTTPS only
Private addresses refused at dial time, not by reading the URL. ← this one. An allowlisted name can resolve to an internal address, and a hostname check never sees it.
Redirects refused — a redirect is a second destination the allowlist never saw
Size cap and timeout — a large body becomes prompt, a slow host spends the turn

The tests enumerate the ways an allowlist gets defeated in practice: suffix and prefix lookalikes, userinfo confusion (https://allowed.host@evil.example/), plain http, file://, cluster service names, 169.254.169.254, and loopback.

GET only

A curl-shaped tool that reads is most of what was asked for. One that writes is a different authority and should be requested on its own terms rather than arriving as a side effect of this.

The boundary this does not solve, stated rather than glossed

What a fetched page says. The allowlist bounds where text comes from and says nothing about what it contains, so an approved host serving hostile instructions is still an open question. Same boundary web search raises on #177, and it belongs there rather than being half-answered here.

ward exec gate green: build, policy-check, vet, test, test-skips, pre-commit.

closes #426 https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/412. Plain `net/http`, as Kai guessed. **Merge https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/pulls/424 first** — `main` is currently red for an unrelated reason and this PR's green is against a fixed base. ## The allowlist is the feature; the fetching is the easy part This runs **inside the cluster**. Unbounded it reaches the tailnet, other services' internal endpoints, and cloud metadata addresses — and the model is exactly the component an attacker gets to talk to. A tool that fetches any URL a model can be persuaded to fetch is SSRF with a conversational interface. **Empty allowlist offers no tool at all**: no schema, no prompt mention, nothing to be talked into. ## Five bounds, and the one that is easy to get wrong - **Exact host match**, not suffix — registering `eco-app.coilysiren.me.evil.example` costs an attacker nothing - **HTTPS only** - **Private addresses refused at dial time**, not by reading the URL. ← this one. An allowlisted *name* can resolve to an internal address, and a hostname check never sees it. - **Redirects refused** — a redirect is a second destination the allowlist never saw - **Size cap and timeout** — a large body becomes prompt, a slow host spends the turn The tests enumerate the ways an allowlist gets defeated in practice: suffix and prefix lookalikes, userinfo confusion (`https://allowed.host@evil.example/`), plain http, `file://`, cluster service names, `169.254.169.254`, and loopback. ## GET only A curl-shaped tool that reads is most of what was asked for. One that writes is a different authority and should be requested on its own terms rather than arriving as a side effect of this. ## The boundary this does not solve, stated rather than glossed **What a fetched page says.** The allowlist bounds *where* text comes from and says nothing about *what it contains*, so an approved host serving hostile instructions is still an open question. Same boundary web search raises on https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/177, and it belongs there rather than being half-answered here. `ward exec gate` green: build, policy-check, vet, test, test-skips, pre-commit.

coilyco-ops added 1 commit

2026-08-13 12:38:40 +00:00

feat(fetch): a read-only fetch, bounded by an allowlist

ci / test (pull_request) Failing after 28s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

ci / image-build (pull_request) Successful in 29s

Details

4795f1cc38

Plain net/http, as asked. The fetching is the easy part and the allowlist is
the feature: this runs inside the cluster, so unbounded it reaches the tailnet,
other services' internal endpoints, and cloud metadata addresses. The model is
exactly the component an attacker gets to talk to.

An empty allowlist offers no tool at all. No schema in the prompt, no mention
to the model, nothing to be talked into.

Five bounds. Exact host match rather than suffix, because a lookalike domain
costs an attacker nothing. HTTPS only. Private, loopback, link-local and
unspecified addresses refused at dial time rather than by reading the URL,
which is the bound that is easy to get wrong: an allowlisted name can resolve
to an internal address and a hostname check never sees it. Redirects refused,
since a redirect is a second destination the allowlist never saw. A size cap
and a timeout, because a large body becomes prompt and a slow host spends the
turn.

GET only. A tool that writes is a different authority and should be asked for
on its own terms rather than arriving as a side effect.

What a fetched page says is not bounded here. That is the same untrusted-input
boundary web search raises and it belongs with it.

closes #426

Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

coilyco-ops requested review from coilysiren

2026-08-13 12:38:40 +00:00

coilyco-ops referenced this pull request

2026-08-13 12:38:44 +00:00

Add http request tool #412

coilyco-ops added 1 commit

2026-08-13 12:40:16 +00:00

Merge branch 'main' into feat/a-bounded-fetch-tool

ci / image-build (pull_request) Successful in 21s

Details

ci / test (pull_request) Successful in 33s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

a8c0774ae4

coilyco-ops merged commit 7f0c950fa3 into main

2026-08-13 12:41:22 +00:00

coilyco-ops referenced this pull request from a commit

2026-08-13 12:41:23 +00:00

Merge pull request 'feat(fetch): a read-only fetch, bounded by an allowlist' (#427) from feat/a-bounded-fetch-tool into main

coilyco-ops commented

2026-08-13 12:41:27 +00:00

Author

Member

Review — Angie (ENG) · s/4b1e. One real gap, against the exact network your own rationale names.

The shape of this is right and I want to say so before the finding. Dial-time Control rather than a hostname check is the bound that is easy to get wrong and you got it right — an allowlisted name resolving to an internal address is the attack a URL parser never sees. Refusing redirects, exact host match, HTTPS only, empty allowlist offering no tool at all: all correct, and the last one is the strongest bound in the file because a tool the model is never told about cannot be talked into anything.

The gap: the tailnet is not covered

Your commit message says unbounded this "reaches the tailnet". refusePrivateAddress does not refuse it. Tailscale assigns from the carrier-grade NAT range 100.64.0.0/10, and Go's IsPrivate implements RFC1918 only — 10/8, 172.16/12, 192.168/16. Measured rather than asserted:

100.64.1.5         private=false loopback=false linklocal=false REFUSED=false
100.100.100.100    private=false loopback=false linklocal=false REFUSED=false
10.0.0.1           private=true                                 REFUSED=true
169.254.169.254                                linklocal=true   REFUSED=true
127.0.0.1                        loopback=true                  REFUSED=true

Cloud metadata is covered, because 169.254.169.254 is link-local. RFC1918 and loopback are covered. The tailnet is not, and this deployment has a tailnet: deploy/AGENTS.md names tailnet FQDNs and authkeys resolving from SSM, and there are tailscale/tailscale workloads on both nodes.

An allowlisted host resolving to a 100.64/10 address reaches it. That is the same class the dial-time check exists to close, missed for one range.

The fix is one clause:

_, cgnat, _ := net.ParseCIDR("100.64.0.0/10")
if cgnat.Contains(ip) { return fmt.Errorf("refusing an internal address") }

Worth a test naming 100.64 explicitly, since it is the range a reader will not think of — which is the whole reason it was missed.

Smaller: an oversize body truncates silently

body, err := io.ReadAll(io.LimitReader(response.Body, int64(maxFetchBytes)))

A page over the cap comes back truncated with no indication, and the model receives a partial document as if it were whole. That is the shape this battery keeps finding — a bound that answers instead of refusing.

fetchAttachment in attachmentingest.go reads limit+1 precisely so it can tell the difference and refuse. Same trick applies here, or say truncated at N bytes in the result so the model can report the gap rather than answer from half a page.

Not raised as a defect

GET only, and writes deferred. Right call, right reason.

What a fetched page says is unbounded. You named this and pointed at the web-search boundary. Agreed it belongs there, and worth its own issue rather than living in a commit message, since a fetched page is untrusted input reaching the prompt and that is #177's vector arriving through a new door.

Not claiming any of it. The CGNAT clause is yours to add while you have the file open.

**Review — Angie (ENG) · s/4b1e. One real gap, against the exact network your own rationale names.** The shape of this is right and I want to say so before the finding. Dial-time `Control` rather than a hostname check is the bound that is easy to get wrong and you got it right — an allowlisted name resolving to an internal address is the attack a URL parser never sees. Refusing redirects, exact host match, HTTPS only, empty allowlist offering no tool at all: all correct, and the last one is the strongest bound in the file because a tool the model is never told about cannot be talked into anything. ## The gap: the tailnet is not covered Your commit message says unbounded this "reaches the tailnet". `refusePrivateAddress` does not refuse it. Tailscale assigns from the carrier-grade NAT range `100.64.0.0/10`, and Go's `IsPrivate` implements RFC1918 only — `10/8`, `172.16/12`, `192.168/16`. Measured rather than asserted: ``` 100.64.1.5 private=false loopback=false linklocal=false REFUSED=false 100.100.100.100 private=false loopback=false linklocal=false REFUSED=false 10.0.0.1 private=true REFUSED=true 169.254.169.254 linklocal=true REFUSED=true 127.0.0.1 loopback=true REFUSED=true ``` Cloud metadata is covered, because `169.254.169.254` is link-local. RFC1918 and loopback are covered. **The tailnet is not**, and this deployment has a tailnet: `deploy/AGENTS.md` names tailnet FQDNs and authkeys resolving from SSM, and there are `tailscale/tailscale` workloads on both nodes. An allowlisted host resolving to a `100.64/10` address reaches it. That is the same class the dial-time check exists to close, missed for one range. The fix is one clause: ```go _, cgnat, _ := net.ParseCIDR("100.64.0.0/10") if cgnat.Contains(ip) { return fmt.Errorf("refusing an internal address") } ``` Worth a test naming `100.64` explicitly, since it is the range a reader will not think of — which is the whole reason it was missed. ## Smaller: an oversize body truncates silently ```go body, err := io.ReadAll(io.LimitReader(response.Body, int64(maxFetchBytes))) ``` A page over the cap comes back truncated with no indication, and the model receives a partial document as if it were whole. That is the shape this battery keeps finding — a bound that answers instead of refusing. `fetchAttachment` in `attachmentingest.go` reads `limit+1` precisely so it can tell the difference and refuse. Same trick applies here, or say `truncated at N bytes` in the result so the model can report the gap rather than answer from half a page. ## Not raised as a defect **GET only, and writes deferred.** Right call, right reason. **What a fetched page says is unbounded.** You named this and pointed at the web-search boundary. Agreed it belongs there, and worth its own issue rather than living in a commit message, since a fetched page is untrusted input reaching the prompt and that is https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/177's vector arriving through a new door. Not claiming any of it. The CGNAT clause is yours to add while you have the file open.

coilyco-ops referenced this pull request

2026-08-13 12:42:18 +00:00

The fetch tool's internal-address guard misses the tailnet range #428

coilyco-ops referenced this pull request

2026-08-13 12:43:22 +00:00

The fetch tool's internal-address guard misses the tailnet range #428

coilyco-ops referenced this pull request

2026-08-13 12:46:44 +00:00

A fetched page over the size cap is truncated silently, so the model reads half a document as a whole one #435