fetch_url decodes a gif as text, so allowlisting a media host buys nothing #1119

New issue

Closed

opened 2026-08-22 23:08:16 +00:00 by coilyco-ops · 0 comments

coilyco-ops commented

2026-08-22 23:08:16 +00:00

Member

Split out of #1029, whose design note names the two shapes but whose allowlist half is deploy-owned.

The gap

fetch_url reads up to SIRENS_ECHO_FETCH_BYTES of any response and returns it as text regardless of content type. For a gif that means thousands of NUL bytes with the invalid runes stripped. Removing the fix from the tool makes the test print it:

result = "200\nGIF89a\x00\x00\x00\x00\x00..." (4 KB of it)

So allowlisting tenor, giphy, or imgur without this buys nothing. The model spends a round to receive garbage and still cannot tell whether the URL is a real asset, which is the exact waste #940 measured on the fetch path from the other direction.

What it should do

Describe the URL rather than decode it. Discord renders a bare media URL as an embed, so what the caller needs is whether this URL is a real asset of the right kind, not its bytes. #1029's own test for an entry is "does fetching it return the actual thing", and a type plus a length answers that.

Acceptance

A non-text content type returns its type and length and no body.
A page still comes back as a page, so this does not trade one broken answer for another.
The readable set is matched by shape, text/* plus +json and +xml by suffix, rather than by an enumeration a new vendor type falls out of.
An absent content type is read, because that is what a plain page used to send.

Not this issue

SIRENS_ECHO_FETCH_HOSTS is a deployment variable, so the curated list from #1029 is deploy's to add.

Refs #1029, #996, #940

Split out of #1029, whose design note names the two shapes but whose allowlist half is deploy-owned. ## The gap `fetch_url` reads up to `SIRENS_ECHO_FETCH_BYTES` of any response and returns it as text **regardless of content type**. For a gif that means thousands of NUL bytes with the invalid runes stripped. Removing the fix from the tool makes the test print it: ``` result = "200\nGIF89a\x00\x00\x00\x00\x00..." (4 KB of it) ``` So allowlisting tenor, giphy, or imgur without this buys nothing. The model spends a round to receive garbage and still cannot tell whether the URL is a real asset, which is the exact waste #940 measured on the fetch path from the other direction. ## What it should do Describe the URL rather than decode it. Discord renders a bare media URL as an embed, so what the caller needs is **whether this URL is a real asset of the right kind**, not its bytes. #1029's own test for an entry is "does fetching it return the actual thing", and a type plus a length answers that. ## Acceptance * A non-text content type returns its type and length and no body. * A page still comes back as a page, so this does not trade one broken answer for another. * The readable set is matched by shape, `text/*` plus `+json` and `+xml` by suffix, rather than by an enumeration a new vendor type falls out of. * An absent content type is read, because that is what a plain page used to send. ## Not this issue `SIRENS_ECHO_FETCH_HOSTS` is a deployment variable, so the curated list from #1029 is deploy's to add. Refs #1029, #996, #940

coilyco-ops referenced this issue from a pull request that will close it,

2026-08-22 23:08:38 +00:00

feat(fetch): describe a media URL instead of decoding it as text #1120

coilyco-ops referenced this issue

2026-08-22 23:08:54 +00:00

allow pulling actual gifs from tenor.com #996

coilyco-ops referenced this issue

2026-08-22 23:09:18 +00:00

allow pulling actual gifs from tenor.com #996

coilyco-ops closed this issue

2026-08-22 23:13:06 +00:00

coilyco-ops referenced this issue