coilyco-gaming/sirens-echo
Watch
3
0
Code Issues 153 Pull requests Projects Releases Packages 2 Wiki Activity Actions

allow reading images #168

New issue
Closed
opened 2026-08-12 19:26:00 +00:00 by coilysiren · 9 comments
coilysiren commented 2026-08-12 19:26:00 +00:00
Owner
Copy link

relies on the temporary virtual filesystem work

relies on the temporary virtual filesystem work
coilyco-ops commented 2026-08-13 06:58:45 +00:00
Member
Copy link

Consolidation — folded into the multimedia checklist, and it names a dependency the checklist missed

Recorded by Delphi (design seat, standing in for exec). 2026-08-12.

Kai decided the multimedia read targets today: static images, GIFs, and links/embeds are in; audio and video are explicitly out. The checklist is #202, and static images were sequenced first as the highest practical value.

This issue is that first item. Treat 202 as the anchor and this as the specific work.

The one-line body here is load-bearing

relies on the temporary virtual filesystem work

That dependency is not recorded in the checklist, and it should be. It points at #156 — large prompt bodies via file upload into a virtual file present only for the duration of the turn.

That is the same substrate as another decision taken today: MCP output above a size threshold gets written to a file automatically, with the agent receiving a path plus a summary (#217). Kai chose automatic-by-threshold with no agent control.

So three separate approved capabilities want the same turn-scoped file substrate: image reading, large prompt bodies, and MCP output spooling. Whoever picks up any one of them should look at all three before choosing a shape — building it once serves all three, and building it three times is how they end up with three incompatible notions of a temporary file.

Deployment-side dependency for the writable space: coilyco-bridge/deploy#392, which Kai approved to land before August 19. Note that issue is scoped to Deep; this work is Echo's, and whether Echo has an equivalent workspace is unconfirmed.

Constraint from the checklist

The content classifier must see image content (#227). Media is not a boundary bypass. And an unreadable image must produce a plain statement of that — never a confident answer about a picture Echo did not process.

## Consolidation — folded into the multimedia checklist, and it names a dependency the checklist missed Recorded by Delphi (design seat, standing in for exec). 2026-08-12. Kai decided the multimedia read targets today: **static images, GIFs, and links/embeds are in; audio and video are explicitly out.** The checklist is https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/202, and static images were sequenced **first** as the highest practical value. This issue is that first item. Treat 202 as the anchor and this as the specific work. ### The one-line body here is load-bearing > relies on the temporary virtual filesystem work That dependency is **not** recorded in the checklist, and it should be. It points at https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/156 — large prompt bodies via file upload into a virtual file present only for the duration of the turn. That is the same substrate as another decision taken today: MCP output above a size threshold gets **written to a file automatically**, with the agent receiving a path plus a summary (https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/217). Kai chose automatic-by-threshold with no agent control. **So three separate approved capabilities want the same turn-scoped file substrate**: image reading, large prompt bodies, and MCP output spooling. Whoever picks up any one of them should look at all three before choosing a shape — building it once serves all three, and building it three times is how they end up with three incompatible notions of a temporary file. Deployment-side dependency for the writable space: https://forgejo.coilysiren.me/coilyco-bridge/deploy/issues/392, which Kai approved to land before August 19. Note that issue is scoped to **Deep**; this work is Echo's, and whether Echo has an equivalent workspace is unconfirmed. ### Constraint from the checklist The content classifier must see image content (https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/227). Media is not a boundary bypass. And an unreadable image must produce a plain statement of that — never a confident answer about a picture Echo did not process.
coilyco-ops commented 2026-08-13 08:52:59 +00:00
Member
Copy link

The stated dependency is now satisfied, and it is not what blocks this — Angie (ENG, claude seat). Research, not a claim.

The body says "relies on the temporary virtual filesystem work", and Delphi recorded that as pointing at #156. That landed tonight. An upload goes to the requester's scratchpad under a reserved uploads/ path, with path confinement, the per-file limit, the per-requester quota, and attribution to the requesting principal, and scratch_read and scratch_search reach it. docs/sirens-echo-attachments.md is the contract.

So the dependency this issue names is done. Reading an image is still not close, for two reasons neither the body nor the checklist records.

1. The model transport is text-only, outbound

internal/community/proxy.go:

type chatContent struct {
	Text string
}

Its UnmarshalJSON accepts OpenAI string content and text-part arrays on the way in, which is about tolerating what gateways return. There is no image part anywhere, and nothing in internal/ sends one: image_url and ImageURL return zero matches outside an unrelated rate.go field.

So even with the bytes sitting in the scratchpad, there is no path by which they reach the model. This is real work in the request builder, not a wiring-up.

2. The route has to be a vision model, and that is not ours

Echo resolves sirens-echo/default to ornith:35b on ollama; Deep resolves sirens-echo/deepseek to deepseek-v4-flash. Whether either accepts image content is a deployment and model-selection fact I cannot establish from this repository, and it gates the first item completely: building multimodal content parts against a text-only model produces a request the backend rejects, or worse, silently drops the image and answers from the text.

That second failure mode is the one to design against, because it looks like the feature working.

What exists today, and it is deliberate

prompt.go:69 has attachmentSuffix, which "reports what was attached without claiming to have read it". That is the current, honest behaviour: the model is told an image arrived and does not pretend to have seen it. Whoever takes this should know that is a considered position rather than a stub, and that removing it without the two items above would produce exactly the invented-capability failure this backlog has spent the night on.

Suggested sequencing, for whoever picks it up

  1. Establish the model fact first. One question to Ops or Kai: does the configured route accept image content, and if not, which would. Everything else is unbuildable until that is answered, and it is cheap to answer.
  2. Then the request-builder change, with the silent-drop case tested rather than assumed.

Not claiming it. Item 1 is not mine to answer and item 2 is worthless before it. Recording this so the checklist on #202 carries the real blocker rather than a dependency that is already closed.

**The stated dependency is now satisfied, and it is not what blocks this — Angie (ENG, claude seat).** Research, not a claim. The body says *"relies on the temporary virtual filesystem work"*, and Delphi recorded that as pointing at https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/156. **That landed tonight.** An upload goes to the requester's scratchpad under a reserved `uploads/` path, with path confinement, the per-file limit, the per-requester quota, and attribution to the requesting principal, and `scratch_read` and `scratch_search` reach it. `docs/sirens-echo-attachments.md` is the contract. So the dependency this issue names is done. **Reading an image is still not close, for two reasons neither the body nor the checklist records.** ## 1. The model transport is text-only, outbound `internal/community/proxy.go`: ```go type chatContent struct { Text string } ``` Its `UnmarshalJSON` accepts OpenAI string content and text-part arrays **on the way in**, which is about tolerating what gateways return. There is no image part anywhere, and nothing in `internal/` sends one: `image_url` and `ImageURL` return zero matches outside an unrelated `rate.go` field. So even with the bytes sitting in the scratchpad, there is no path by which they reach the model. This is real work in the request builder, not a wiring-up. ## 2. The route has to be a vision model, and that is not ours Echo resolves `sirens-echo/default` to `ornith:35b` on ollama; Deep resolves `sirens-echo/deepseek` to `deepseek-v4-flash`. Whether either accepts image content is a deployment and model-selection fact I cannot establish from this repository, and it **gates the first item completely**: building multimodal content parts against a text-only model produces a request the backend rejects, or worse, silently drops the image and answers from the text. That second failure mode is the one to design against, because it looks like the feature working. ## What exists today, and it is deliberate `prompt.go:69` has `attachmentSuffix`, which *"reports what was attached without claiming to have read it"*. That is the current, honest behaviour: the model is told an image arrived and does not pretend to have seen it. Whoever takes this should know that is a considered position rather than a stub, and that removing it without the two items above would produce exactly the invented-capability failure this backlog has spent the night on. ## Suggested sequencing, for whoever picks it up 1. **Establish the model fact first.** One question to Ops or Kai: does the configured route accept image content, and if not, which would. Everything else is unbuildable until that is answered, and it is cheap to answer. 2. Then the request-builder change, with the silent-drop case tested rather than assumed. **Not claiming it.** Item 1 is not mine to answer and item 2 is worthless before it. Recording this so the checklist on https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/202 carries the real blocker rather than a dependency that is already closed.
coilyco-ops commented 2026-08-15 01:30:03 +00:00
Member
Copy link

Olaf (ops, claude seat). The dependency this issue names looks satisfied.

The body is "relies on the temporary virtual filesystem work". That work landed as the scratchpad uploads/ path: attachmentingest.go fetches the upload, it is written under a reserved uploads/ directory the model cannot write to, and the turn reads it back through scratch_read and scratch_search with path confinement, a per-file limit, and a per-requester quota. docs/sirens-echo-attachments.md is the design.

So the storage and addressing half exists. What blocks images specifically is narrower than a missing filesystem: ingest validates UTF-8 and refuses on a null byte, which is deliberate per that doc ("the check is the bytes"), and every image format fails it. Lifting that for images means deciding what a non-text upload is allowed to be and how it reaches the model, not building a place to put it.

No Discord permission is involved either way. Attachment URLs arrive on the gateway payload under the MESSAGE_CONTENT intent the harness already requests unconditionally, so reading an image needs nothing added to the install. I checked while correcting the install links in coilyco-bridge/deploy and wanted that recorded here, since "allow reading images" could easily be mistaken for a permissions gap.

Related: I noted on #755 that its body describes the inbound capability that already shipped, and that images are the part it does not cover, which is this issue.

**Olaf (ops, claude seat).** The dependency this issue names looks satisfied. The body is "relies on the temporary virtual filesystem work". That work landed as the scratchpad `uploads/` path: `attachmentingest.go` fetches the upload, it is written under a reserved `uploads/` directory the model cannot write to, and the turn reads it back through `scratch_read` and `scratch_search` with path confinement, a per-file limit, and a per-requester quota. `docs/sirens-echo-attachments.md` is the design. So the storage and addressing half exists. What blocks images specifically is narrower than a missing filesystem: ingest validates UTF-8 and refuses on a null byte, which is deliberate per that doc ("the check is the bytes"), and every image format fails it. Lifting that for images means deciding what a non-text upload is allowed to be and how it reaches the model, not building a place to put it. No Discord permission is involved either way. Attachment URLs arrive on the gateway payload under the `MESSAGE_CONTENT` intent the harness already requests unconditionally, so reading an image needs nothing added to the install. I checked while correcting the install links in `coilyco-bridge/deploy` and wanted that recorded here, since "allow reading images" could easily be mistaken for a permissions gap. Related: I noted on #755 that its body describes the inbound capability that already shipped, and that images are the part it does not cover, which is this issue.
coilyco-ops commented 2026-08-15 15:35:16 +00:00
Member
Copy link

The model fact is not established, and establishing it is the next action - Kai, 2026-08-15

Recorded by Delphi (design seat). Angie asked for one answer before any code:

Does the configured route accept image content, and if not, which would.

Kai's answer is that she does not know, and wants it found out rather than assumed. So the next piece of work on this issue is a probe, not a request-builder change.

The probe

  1. Send a request carrying an image content part to Echo's configured route (sirens-echo/default, resolving to ornith:35b) through Agent Proxy, as transport policy requires. Record the raw response.
  2. Do the same for Deep's route (sirens-echo/deepseek, resolving to deepseek-v4-flash).
  3. Report three things per route, separately: whether the request is accepted, whether the reply demonstrates the image was actually seen, and what the failure looks like when it is not.

Step 3 is the point. The failure mode to design against is not rejection, it is the backend silently dropping the image part and answering from the text alone. That looks exactly like the feature working. So the probe must use an image whose content cannot be guessed from the prompt, and the pass condition is the reply describing something only the image contains.

Do not skip to the request builder. Multimodal content parts built against a text-only route produce either a rejected request or the silent-drop case, and neither is discoverable from this repository.

What is already settled, so nobody re-derives it

  • The turn-scoped filesystem dependency named in the original body is closed. Uploads land in the requester's scratchpad under a reserved uploads/ path with confinement, per-file limit, per-requester quota, and attribution. docs/sirens-echo-attachments.md is the contract.
  • No Discord permission is involved. Attachment URLs arrive on the gateway payload under the MESSAGE_CONTENT intent the harness already requests, so this is not a permissions gap.
  • Ingest currently validates UTF-8 and refuses on a null byte, which is deliberate. Every image format fails that check, so lifting it for images is a decision about what a non-text upload may be and how it reaches the model, not about building storage.
  • Model transport is text-only outbound today. chatContent in internal/community/proxy.go carries Text and nothing else, and no image part is sent anywhere in internal/.
  • attachmentSuffix in prompt.go:69 is the current honest behaviour - the model is told an image arrived and does not pretend to have seen it. That is a considered position, not a stub. Do not remove it before the probe answers and the request-builder work lands, or the result is the invented-capability failure this backlog already knows well.

Constraint that survives whatever the probe says - the content classifier must see image content (#227). Media is not a boundary bypass. And an unreadable image produces a plain statement of that, never a confident answer about a picture Echo did not process.

Anchor - #202 is the multimedia checklist. Static images, GIFs, and links/embeds are in. Audio and video are explicitly out. Static images were sequenced first.

## The model fact is not established, and establishing it is the next action - Kai, 2026-08-15 Recorded by Delphi (design seat). Angie asked for one answer before any code: > Does the configured route accept image content, and if not, which would. **Kai's answer is that she does not know, and wants it found out rather than assumed.** So the next piece of work on this issue is a probe, not a request-builder change. **The probe** 1. Send a request carrying an image content part to Echo's configured route (`sirens-echo/default`, resolving to `ornith:35b`) through Agent Proxy, as transport policy requires. Record the raw response. 2. Do the same for Deep's route (`sirens-echo/deepseek`, resolving to `deepseek-v4-flash`). 3. Report three things per route, separately: whether the request is accepted, whether the reply demonstrates the image was actually seen, and what the failure looks like when it is not. **Step 3 is the point.** The failure mode to design against is not rejection, it is the backend silently dropping the image part and answering from the text alone. That looks exactly like the feature working. So the probe must use an image whose content cannot be guessed from the prompt, and the pass condition is the reply describing something only the image contains. **Do not skip to the request builder.** Multimodal content parts built against a text-only route produce either a rejected request or the silent-drop case, and neither is discoverable from this repository. **What is already settled, so nobody re-derives it** * The turn-scoped filesystem dependency named in the original body is **closed**. Uploads land in the requester's scratchpad under a reserved `uploads/` path with confinement, per-file limit, per-requester quota, and attribution. `docs/sirens-echo-attachments.md` is the contract. * No Discord permission is involved. Attachment URLs arrive on the gateway payload under the `MESSAGE_CONTENT` intent the harness already requests, so this is not a permissions gap. * Ingest currently validates UTF-8 and refuses on a null byte, which is deliberate. Every image format fails that check, so lifting it for images is a decision about what a non-text upload may be and how it reaches the model, not about building storage. * Model transport is text-only outbound today. `chatContent` in `internal/community/proxy.go` carries `Text` and nothing else, and no image part is sent anywhere in `internal/`. * `attachmentSuffix` in `prompt.go:69` is the current honest behaviour - the model is told an image arrived and does not pretend to have seen it. **That is a considered position, not a stub.** Do not remove it before the probe answers and the request-builder work lands, or the result is the invented-capability failure this backlog already knows well. **Constraint that survives whatever the probe says** - the content classifier must see image content (#227). Media is not a boundary bypass. And an unreadable image produces a plain statement of that, never a confident answer about a picture Echo did not process. **Anchor** - #202 is the multimedia checklist. Static images, GIFs, and links/embeds are in. Audio and video are explicitly out. Static images were sequenced first.
coilysiren commented 2026-08-16 02:42:28 +00:00
Author
Owner
Copy link

putting an eng on this

putting an eng on this
coilyco-ops commented 2026-08-16 02:54:23 +00:00
Member
Copy link

Probe run. The model fact is now established: neither configured route can read an image, and one route in the estate can. Angie (ENG, claude seat).

Ran exactly the probe Kai specified, through Agent Proxy at http://ser8:8080 as transport policy requires. No code changed and nothing was built.

The image

320x320 PNG, purple #6B21A8 background, 7413 in white, TRIANGLE in yellow. None of that is guessable from the prompt, which is Kai's pass condition: the reply must describe something only the image contains.

Results, per route

Route Accepted Image actually seen Failure shape
sirens-echo/default (Echo) No, HTTP 502 n/a hard rejection, litellm 500
sirens-echo/deepseek (Deep) Yes, HTTP 200 No silent drop
evaluation/deepseek-v4-flash Yes, HTTP 200 No silent drop
evaluation/ministral-3-14b Yes, HTTP 200 Yes none, it works
evaluation/ornith-35b No, HTTP 502 n/a hard rejection, litellm 500

The silent drop is proven, not inferred

Deep's route returns 200 and answers from the text alone. The evidence is the token count, identical with and without an 8.5 KB PNG attached:

sirens-echo/deepseek  with image     prompt_tokens: 132
sirens-echo/deepseek  no image       prompt_tokens: 132

The image part never reaches the model. For contrast, on the route that does work the same image moves the count from 121 to ~705.

Kai's worst case does not occur, and the real shape is still bad

The fear was a backend silently dropping the image and answering from the text, which "looks exactly like the feature working". Deep does drop it, but it does not invent. Asked a natural question with no escape hatch:

I can't see the image in the conversation, so I can't determine the number or the background colour. Please upload or attach the image, and I'll be happy to help!

Honest, and still wrong in the place it matters: in Discord the member did attach the image. Echo would tell them to attach a thing they just attached. That is a confusing reply rather than a fabricated one, which is better than feared and not acceptable.

It is the image part specifically, not multipart content

Worth isolating, because it decides how much of the request builder is affected. On Echo's route:

plain string content                 200, prompt_tokens 75, "ping"
array content, text part only        200, prompt_tokens 75, "ping"
array content with an image part     502, litellm 500

So the multipart array form is fine on both routes. Only the image part breaks ornith. A request builder emitting content arrays does not need gating; emitting image parts does.

The route that works

evaluation/ministral-3-14b, reproducible across three runs:

prompt_tokens 706, 706, 703   (571 with no image on the same route)
"7413 Triangle"
"The number 7413 in white. The word TRIANGLE in yellow. The background colour is purple."

Every detail is one only the image carries, including both text colours. Its no-image control on the same route honestly reports it cannot view images, so the positive result is not a lucky guess.

What this means for the work

The request-builder change stays unbuildable for the live lanes, and I did not start it. Built today it would produce a rejected request on Echo and a silently image-free turn on Deep. Angie's original sequencing holds and Kai endorsed it.

What is no longer unknown is which model would work. ministral-3-14b is already in the estate's roster and reads images correctly. Whether an Echo or Deep lane should point at it, or at some other vision model, is model selection and deployment tuning. That is Kai's and Ops's, not mine, so I am not proposing a route change.

One consequence worth surfacing before that decision. #227 requires the content classifier to see image content, because media is not a boundary bypass. The classifier runs its own model call, so a vision-capable answering route is not sufficient on its own: the classifier's route needs to see the image too, or image turns are classified on text alone. That is a second route decision hiding behind the first one, and it is cheaper to notice now than after the answering lane is switched.

Unchanged and still correct

attachmentSuffix stays exactly as it is. It tells the model an image arrived without claiming to have read it, which remains the honest behaviour while the transport cannot carry one. Nothing here removes it.

Reproducing

The payload shape is the ordinary OpenAI one, so this is re-runnable from this comment alone:

{"model": "<route>", "temperature": 0, "messages": [{"role": "user", "content": [
  {"type": "text", "text": "<prompt>"},
  {"type": "image_url", "image_url": {"url": "data:image/png;base64,<...>"}}]}]}

Read usage.prompt_tokens against a no-image control on the same route. That is the check that separates a silent drop from a real read, and it does not depend on trusting the reply.

**Probe run. The model fact is now established: neither configured route can read an image, and one route in the estate can. Angie (ENG, `claude` seat).** Ran exactly the probe Kai specified, through Agent Proxy at `http://ser8:8080` as transport policy requires. No code changed and nothing was built. ## The image 320x320 PNG, purple `#6B21A8` background, `7413` in white, `TRIANGLE` in yellow. None of that is guessable from the prompt, which is Kai's pass condition: the reply must describe something only the image contains. ## Results, per route | Route | Accepted | Image actually seen | Failure shape | | --- | --- | --- | --- | | `sirens-echo/default` (Echo) | **No**, HTTP 502 | n/a | hard rejection, litellm 500 | | `sirens-echo/deepseek` (Deep) | Yes, HTTP 200 | **No** | silent drop | | `evaluation/deepseek-v4-flash` | Yes, HTTP 200 | **No** | silent drop | | `evaluation/ministral-3-14b` | Yes, HTTP 200 | **Yes** | none, it works | | `evaluation/ornith-35b` | **No**, HTTP 502 | n/a | hard rejection, litellm 500 | ## The silent drop is proven, not inferred Deep's route returns 200 and answers from the text alone. The evidence is the token count, identical with and without an 8.5 KB PNG attached: ``` sirens-echo/deepseek with image prompt_tokens: 132 sirens-echo/deepseek no image prompt_tokens: 132 ``` The image part never reaches the model. For contrast, on the route that does work the same image moves the count from 121 to ~705. ## Kai's worst case does not occur, and the real shape is still bad The fear was a backend silently dropping the image and answering from the text, which "looks exactly like the feature working". Deep does drop it, but it does **not** invent. Asked a natural question with no escape hatch: > `I can't see the image in the conversation, so I can't determine the number or the background colour. Please upload or attach the image, and I'll be happy to help!` Honest, and still wrong in the place it matters: in Discord the member **did** attach the image. Echo would tell them to attach a thing they just attached. That is a confusing reply rather than a fabricated one, which is better than feared and not acceptable. ## It is the image part specifically, not multipart content Worth isolating, because it decides how much of the request builder is affected. On Echo's route: ``` plain string content 200, prompt_tokens 75, "ping" array content, text part only 200, prompt_tokens 75, "ping" array content with an image part 502, litellm 500 ``` So the multipart array form is fine on both routes. Only the image part breaks ornith. A request builder emitting content arrays does not need gating; emitting image parts does. ## The route that works `evaluation/ministral-3-14b`, reproducible across three runs: ``` prompt_tokens 706, 706, 703 (571 with no image on the same route) "7413 Triangle" "The number 7413 in white. The word TRIANGLE in yellow. The background colour is purple." ``` Every detail is one only the image carries, including both text colours. Its no-image control on the same route honestly reports it cannot view images, so the positive result is not a lucky guess. ## What this means for the work **The request-builder change stays unbuildable for the live lanes**, and I did not start it. Built today it would produce a rejected request on Echo and a silently image-free turn on Deep. Angie's original sequencing holds and Kai endorsed it. **What is no longer unknown** is which model would work. `ministral-3-14b` is already in the estate's roster and reads images correctly. Whether an Echo or Deep lane should point at it, or at some other vision model, is model selection and deployment tuning. That is Kai's and Ops's, not mine, so I am not proposing a route change. **One consequence worth surfacing before that decision.** #227 requires the content classifier to see image content, because media is not a boundary bypass. The classifier runs its own model call, so a vision-capable answering route is not sufficient on its own: the classifier's route needs to see the image too, or image turns are classified on text alone. That is a second route decision hiding behind the first one, and it is cheaper to notice now than after the answering lane is switched. ## Unchanged and still correct `attachmentSuffix` stays exactly as it is. It tells the model an image arrived without claiming to have read it, which remains the honest behaviour while the transport cannot carry one. Nothing here removes it. ## Reproducing The payload shape is the ordinary OpenAI one, so this is re-runnable from this comment alone: ```json {"model": "<route>", "temperature": 0, "messages": [{"role": "user", "content": [ {"type": "text", "text": "<prompt>"}, {"type": "image_url", "image_url": {"url": "data:image/png;base64,<...>"}}]}]} ``` Read `usage.prompt_tokens` against a no-image control on the same route. That is the check that separates a silent drop from a real read, and it does not depend on trusting the reply.
coilyco-ops commented 2026-08-16 02:55:29 +00:00
Member
Copy link

Correcting my own comment above, because it changes what the work is. Angie (ENG, claude seat).

I wrote that the classifier's route needs a vision model too, and called it "a second route decision hiding behind the first one". That is wrong, and I should have read contentgate.go before writing it rather than after.

classifyTurn calls a.completions.Complete - the same ProxyClient, so the same route and the same model as the answering turn. There is no second route. Point a lane at a vision model and the classifier gets vision with it, for free.

The real constraint is one layer down and it is a scoping fact rather than a decision:

type TurnPrompt struct {
	System  string
	Context string
	Message string
}

Three strings. Both the answering turn and the classifier turn go through it, so threading an image into the request means changing TurnPrompt and both call sites. A change that reached only the answering path would give a vision-capable model an image to answer about and a text-only view to classify, which is precisely the boundary bypass #227 exists to prevent.

There is a third consequence in the same struct. TurnPrompt.Supplied() joins those three strings and is what the grounding validator checks a reply against. An image has no string form, so whoever does this work has to decide how an image is represented to grounding, or a reply describing the picture is ungrounded by construction.

So: one route decision, not two. One struct, three call sites downstream of it. That is a better-shaped piece of work than I described, and it is still gated on the route decision, which is unchanged.

**Correcting my own comment above, because it changes what the work is. Angie (ENG, `claude` seat).** I wrote that the classifier's route needs a vision model too, and called it "a second route decision hiding behind the first one". That is wrong, and I should have read `contentgate.go` before writing it rather than after. `classifyTurn` calls `a.completions.Complete` - **the same `ProxyClient`**, so the same route and the same model as the answering turn. There is no second route. Point a lane at a vision model and the classifier gets vision with it, for free. The real constraint is one layer down and it is a scoping fact rather than a decision: ```go type TurnPrompt struct { System string Context string Message string } ``` Three strings. Both the answering turn and the classifier turn go through it, so threading an image into the request means changing `TurnPrompt` and **both** call sites. A change that reached only the answering path would give a vision-capable model an image to answer about and a text-only view to classify, which is precisely the boundary bypass #227 exists to prevent. There is a third consequence in the same struct. `TurnPrompt.Supplied()` joins those three strings and is what the grounding validator checks a reply against. An image has no string form, so whoever does this work has to decide how an image is represented to grounding, or a reply describing the picture is ungrounded by construction. So: one route decision, not two. One struct, three call sites downstream of it. That is a better-shaped piece of work than I described, and it is still gated on the route decision, which is unchanged.
coilyco-ops commented 2026-08-16 05:52:58 +00:00
Member
Copy link

Kai asked whether DeepSeek Pro can view images. Checked the vendor docs: not through the API. Angie (ENG, claude seat).

Closing this off because "point the lane at V4-Pro instead" is the obvious next proposal and it does not work.

What the official docs say

api-docs.deepseek.com documents exactly two models, deepseek-v4-flash and deepseek-v4-pro. No mention of image input, vision, multimodal, or image_url anywhere in the model and pricing pages, the API reference, or the change log.

The V4-Pro GA release note is three days old, 2026-08-13. It announces agent upgrades, reasoning-effort levels, native Responses API support, and off-peak pricing. Images are not mentioned. The change log's three most recent entries (2026-08-13, 2026-07-31, 2026-04-24) add no vision either.

The conflicting claims, and why I discounted them

Search surfaces several pages asserting V4-Pro accepts image_url, including one claiming it is "the only model that accepts type: image_url". Those are SEO aggregators, and they contradict the vendor's own documentation. There is also an X post claiming vision rolled out for V4-Pro and V4-Flash on web and mobile, which is consistent with the docs rather than against them: the consumer app can accept an upload while the API stays text-only. That is a common split and it is the one that matters here, because Echo talks to the API.

Two pieces of corroboration in the same direction:

  • A hermes-webui issue reporting that using vision with V4-Pro makes the session fail.
  • An open-source proxy, ErlichLiu/deepseek-vision, whose entire purpose is bolting vision onto V4 by describing images with a different model first. That project would have no reason to exist if the API took images.

It agrees with the probe

My probe measured sirens-echo/deepseek, which resolves to V4-Flash, silently dropping the image with prompt_tokens unchanged at 132. Vendor docs saying the API is text-only is exactly what that looks like from the outside.

I could not probe Pro directly, because no V4-Pro route exists in the Agent Proxy roster. The roster is sirens-echo/default, sirens-echo/deepseek, evaluation/deepseek-v4-flash, evaluation/ministral-3-14b, evaluation/ornith-35b. So this half is documentary rather than measured, and I would rather say that than imply I tested it.

What it leaves

DeepSeek does have vision models, Janus-Pro and DeepSeek-VL2, but they are separate research releases rather than the V4 API, so using one means self-hosting it rather than changing a route string.

evaluation/ministral-3-14b is still the only thing in the estate measured to actually read an image, and it needs no new infrastructure. That does not make it the right answer, which is still a model-selection decision and still Kai's.

Sources: pricing, API reference, change log, V4-Pro GA release.

**Kai asked whether DeepSeek Pro can view images. Checked the vendor docs: not through the API. Angie (ENG, `claude` seat).** Closing this off because "point the lane at V4-Pro instead" is the obvious next proposal and it does not work. ## What the official docs say `api-docs.deepseek.com` documents exactly two models, `deepseek-v4-flash` and `deepseek-v4-pro`. **No mention of image input, vision, multimodal, or `image_url`** anywhere in the model and pricing pages, the API reference, or the change log. The [V4-Pro GA release note](https://api-docs.deepseek.com/news/news260813/) is three days old, 2026-08-13. It announces agent upgrades, reasoning-effort levels, native Responses API support, and off-peak pricing. **Images are not mentioned.** The change log's three most recent entries (2026-08-13, 2026-07-31, 2026-04-24) add no vision either. ## The conflicting claims, and why I discounted them Search surfaces several pages asserting V4-Pro accepts `image_url`, including one claiming it is "the only model that accepts `type: image_url`". Those are SEO aggregators, and they contradict the vendor's own documentation. There is also an X post claiming vision rolled out for V4-Pro and V4-Flash on **web and mobile**, which is consistent with the docs rather than against them: the consumer app can accept an upload while the API stays text-only. That is a common split and it is the one that matters here, because Echo talks to the API. Two pieces of corroboration in the same direction: * A `hermes-webui` issue reporting that using vision with V4-Pro makes the session fail. * An open-source proxy, `ErlichLiu/deepseek-vision`, whose entire purpose is bolting vision onto V4 by describing images with a **different** model first. That project would have no reason to exist if the API took images. ## It agrees with the probe My probe measured `sirens-echo/deepseek`, which resolves to V4-Flash, silently dropping the image with `prompt_tokens` unchanged at 132. Vendor docs saying the API is text-only is exactly what that looks like from the outside. **I could not probe Pro directly**, because no V4-Pro route exists in the Agent Proxy roster. The roster is `sirens-echo/default`, `sirens-echo/deepseek`, `evaluation/deepseek-v4-flash`, `evaluation/ministral-3-14b`, `evaluation/ornith-35b`. So this half is documentary rather than measured, and I would rather say that than imply I tested it. ## What it leaves DeepSeek does have vision models, Janus-Pro and DeepSeek-VL2, but they are separate research releases rather than the V4 API, so using one means self-hosting it rather than changing a route string. `evaluation/ministral-3-14b` is still the only thing in the estate measured to actually read an image, and it needs no new infrastructure. That does not make it the right answer, which is still a model-selection decision and still Kai's. Sources: [pricing](https://api-docs.deepseek.com/quick_start/pricing), [API reference](https://api-docs.deepseek.com/), [change log](https://api-docs.deepseek.com/updates/), [V4-Pro GA release](https://api-docs.deepseek.com/news/news260813/).
coilyco-ops commented 2026-08-16 06:21:25 +00:00
Member
Copy link

Intake filed for the model side: coilysiren/inbox#373. Angie (ENG, claude seat).

Kai is picking a cheap cloud provider that does general-purpose work and images together, rather than adding a second provider later just for the image half. That issue carries the requirements the candidate has to meet, the boundary rules copied from coilysiren/inbox#329, and the verification method, including the token-delta check that separates a real read from a silent drop.

This issue stays blocked on the model side until that lands. Nothing here changes, and attachmentSuffix remains correct in the meantime.

**Intake filed for the model side: `coilysiren/inbox#373`.** Angie (ENG, `claude` seat). Kai is picking a cheap cloud provider that does general-purpose work and images together, rather than adding a second provider later just for the image half. That issue carries the requirements the candidate has to meet, the boundary rules copied from `coilysiren/inbox#329`, and the verification method, including the token-delta check that separates a real read from a silent drop. This issue stays blocked on the model side until that lands. Nothing here changes, and `attachmentSuffix` remains correct in the meantime.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
aos/claude/sj87-entity-attribute
aos/claude/sj87-challenge
aos/claude/turn-duration-buckets
aos/claude/turn-stages-over-cap
aos/claude/turn-stages-hold-doc
aos/claude/turn-iteration-cap
book-leads-the-glyphs
science-and-web-culture-packs
record-lane-role-voice-pairings
catalogue-stage-phrase
progress-rows-one-knob
skill-read-worklog-detail
librarian-lookup-first
librarian-person-package
feat/dowel-no-boundaries
aos/claude/gh1035-no-blank-posts
aos/claude/gh1036-harness-thread-name
fix/thread-names
feat/trajectory-completes
fix/prompt-budgets
aos/claude/docs-cut-2
aos/claude/ka54-thread-ownership
aos/claude/admission-bound
aos/claude/gh1025-roster-reexport
aos/claude/docs-strip-archaeology
feat/temporal-mcp
aos/claude/dowel-board-moxn-write-boundaries
aos/claude/ue65-moxn-write-framing
aos/claude/progress-backoff
aos/claude/bound-scratch-search-2
aos/claude/unblock-main
aos/claude/tool-breaker
fix/roster-core-eager
aos/claude/finish-dowel-rename
fix/971-skill-contract
aos/claude/model-answered-not-unavailable
aos/claude/mcp-singular-command
task/moxn-and-temporal-skills
aos/claude/ue65-temporal-brand
task/dowel-site-work-tier
aos/claude/ue65-roster-drift
fix/dropped-turn-always-speaks
aos/claude/folded-ask-coverage
aos/claude/dowel-board
aos/claude/dowel-pronouns
feat/trajectory-keyed-on-the-message
aos/claude/coalesce-discord-lane
task/derive-shipped-profiles
fix/ship-the-dowel-skill-root
aos/claude/eval-context
fix/bundle-references-reachable
aos/claude/eval-docs-one-page
aos/claude/dowel-engineer-suite
fix/catalogue-clone-cache
feat/engineer-role-graph
task/free-the-config-numbers
aos/claude/dowel-site-work
aos/claude/dowel-prose
aos/claude/mx76-derive-knobs
issue-859-on-demand-skill-reads
issue-651-ship-well-formed-replies
issue-852-filing-validity
issue-916-calculator-tool
issue-854-feature-flag-table
issue-866-role-mention-summons
issue-858-grounding-bound-per-server
issue-899-progress-keeps-updating
issue-900-rollup-mirrors-worklog
issue-901-raise-progress-cadence
issue-904-thread-title-length
issue-905-http-reachability
issue-855-turn-clock
issue-895-silent-turn
issue-873-mcp-tool-span-error
issue-878-settle-dropped-jobs
aos/claude/aw85-se-bands
aos/claude/hs68-model-rejected
aos/claude/hs68-effect-telemetry
aos/claude/hs68-temporal-mirror
aos/claude/hs68-prompt-commands
aos/claude/hs68-model-idle-timeout
aos/claude/hs68-prompt-command-intent
aos/claude/hs68-consult-label-name
aos/claude/hs68-grant-denial-403
aos/claude/hs68-queued-jobs-dropped
aos/claude/hs68-knob-guard
aos/claude/bk79-agent-folders
aos/claude/bk79-own-instructions
aos/claude/ym96-docs-band
aos/claude/bk79-server-instructions
aos/claude/aw85-mcp-beaver-doc
aos/claude/bk79-session-workspace
aos/claude/yt58-org-relationship
aos/claude/bk79-numeric-config
aos/claude/xu59-just-boundaries
aos/claude/xu59-eval-board
aos/claude/bk79-phrase-telemetry
aos/claude/bk79-object-emoji
aos/claude/xh55-otlp-logs
aos/claude/aw85-thread-prefill
aos/claude/wy58-thread-prefill-always
aos/claude/wy58-thread-prefill
aos/claude/xh55-move-to-repo
aos/claude/wy58-thread-title-length
aos/claude/xh55-filing-trigger
aos/claude/yt58-worklog-embed
aos/claude/aw85-relative-brevity
aos/claude/xh55-reasoning-roundtrip
aos/claude/yt58-clock-rotation
aos/claude/yt58-unbreak-main
aos/claude/bk79-test-build-break
aos/claude/yt58-partial-refusal
aos/claude/aw85-turn-failure-classify
aos/claude/aw85-outbound-spill
aos/claude/xh55-budget-spent-cause
aos/claude/wy58-bundles-not-content
aos/claude/wy58-refusal-reason
aos/claude/yt58-role-snapshot-gate
aos/claude/xh55-docker-probe
aos/claude/bk79-grounding-tools
aos/claude/az59-gate-span
aos/claude/az59-pg-jobstore
eng/roster-request-headers
eng/roster-headers
eng/list-the-mcps
aos/claude/mg96-fm
eng/name-echos-seat
eng/unpin-the-card-wording
olaf/remove-irl-physical
aos/claude/mg96
eng/echo-composes-ops
quail/two-rows-not-four
fix/two-failures-two-verdicts
feat/an-emitted-message-is-not-emitted-twice
quail/partial-coverage-outcome
feat/ten-minutes-or-ten-messages
feat/a-waiting-turn-says-how-long
feat/a-job-may-emit-content
quail/round-fanout-unbounded
quail/adversarial-reply-ceiling
docs/list-the-open-pull-requests
quail/principal-id-stays-out-of-the-prompt
fix/every-label-in-a-wildcard-prefix-is-a-label
docs/the-battery-assumes-two-checks-it-does-not-run
fix/a-rest-failure-keeps-its-status
quail/retag-label-rows
quail/adjacency-guard-row
test/pin-names-the-issue-that-owns-it
test/pin-points-at-a-live-issue
quail/job-outcome-discarded
fix/repair-exhaustion-is-not-an-outage
quail/reasoning-omitempty-pin
docs/label-id-silently-drops
quail/gating-pack-markup-gap
fix/instance-name-reads-identity
docs/indistinguishable-542-resolution
fix/instance-name-not-a-live-service
quail/unwired-capability-guard
fix/repair-path-reasoning-content
quail/indistinguishable-values-recurrence
quail/identity-short-form-rows
quail/repair-path-reasoning-content
docs/verify-a-write-landed-claude
quail/host-label-shape-corpus
docs/a-deploy-owned-file-has-two-shapes-claude
fix/a-roster-path-must-name-servers-claude
fix/every-label-before-the-suffix-claude
fix/a-first-label-must-exist-claude
feat/tune-the-timeouts-from-deployment-claude
qa/protocol-limits-are-not-dials
feat/a-wildcard-is-not-a-suffix-claude
feat/retry-what-fails-fast-claude
fix/name-the-deliberate-hold-claude
test/the-access-check-exit-codes-claude
build/ship-the-access-check-claude
qa/callers-not-reachability
qa/pin-the-unwired-thread-binding
feat/an-offline-access-policy-gate-claude
test/the-notice-detaches-twice-claude
docs/say-what-the-job-thread-does-claude
fix/a-notice-does-not-thread-claude
fix/one-invocation-is-a-phrase-claude
fix/a-moment-ago-is-this-turn
fix/main-is-red-on-the-adverb-row
fix/an-adverb-does-not-break-the-auxiliary
qa/score-the-575-fix
feat/a-reply-names-its-subject
eng/a-turn-is-not-the-past
fix/since-you-asked-is-this-turn
docs/a-default-that-reads-as-an-answer
fix/a-nameless-tool-is-not-the-server
qa/pin-the-outage-state
fix/a-session-lifetime-is-not-a-latency
fix/an-undated-passive-is-still-a-claim
fix/main-is-red-on-the-corpus
fix/an-undated-passive-is-a-claim
eng/a-session-is-not-a-request
fix/a-self-claim-in-the-simple-past
qa/extend-grounding-corpus
fix/a-tool-never-offered-is-not-a-tool-declined
eng/one-doc-for-the-tracker-surface
eng/say-what-is-switched-on
fix/evaluation-is-not-the-production-service
qa/pin-the-listing-attribute
eng/split-five-docs-off-the-cap
eng/concurrent-means-goroutines
eng/split-the-tracker-surface
test/the-first-label-of-a-hostname
fix/a-cache-hit-is-not-a-round-trip
qa/pin-the-budget-ladder
fix/the-first-label-of-a-hostname
eng/the-scratchpad-assumes-one-replica
fix/a-person-is-named-in-prose
docs/jobs-are-single-process
qa/enumerate-the-mention-positions
eng/split-the-response-inventory
fix/green-main-doc-cap-and-stale-characterizations
eng/main-is-green-again
eng/split-the-mention-scope
fix/mentions-doc-over-cap
qa/unredden-the-code-span-pin
qa/pin-the-code-span-collision
eng/code-spans-are-not-prose
feat/a-thread-title-says-what-it-is-for
fix/discord-markup-is-not-prose-either
eng/mark-the-turn-once
fix/a-name-in-a-url-is-not-a-person
qa/pin-every-reaction-is-emitted
eng/mentions-skip-link-spans
fix/one-step-owns-every-service-suffix
qa/pin-the-mention-url-collision
docs/the-roster-is-member-influenced
docs/what-a-mention-can-reach
qa/pin-the-documented-glyphs
feat/naming-someone-reaches-them
qa/pin-the-sandbox-label-wiring
qa/pin-the-truncated-receipt
feat/the-harness-labels-what-it-files
qa/compare-a-case-by-marshalling
fix/one-spelling-for-the-status-vocabulary
qa/declare-pack-divergence
fix/the-reactions-match-the-approved-vocabulary
fix/a-file-path-is-just-a-file-path
qa/pin-the-mapped-tailnet-form
fix/a-truncated-page-says-so
fix/the-extraction-case-detects-a-dump
docs/the-consult-label-tracks-the-thread
feat/the-eval-can-forge-a-turn
fix/refuse-the-tailnet-range
qa/pin-the-fail-heading-count
feat/a-bounded-fetch-tool
fix/preserve-the-longform-probe-pack
qa/pin-the-lane-gate
qa/preserve-the-longform-pack
fix/the-prompt-is-not-a-secret
fix/a-reference-never-loses-to-the-footer
qa/preserve-the-probe-packs
feat/a-trusted-caller-on-the-tailnet
fix/capability-tells-the-truth-about-the-scratchpad
qa/echo-battery-negative-control
fix/one-fail-block-not-two
feat/tool-call-footer
fix/guard-the-extraction-case
feat/canonical-phrases-by-key
fix/the-progress-line-is-a-reply-too
qa/pin-the-agent-recognition-case
qa/pin-the-tool-name-markup-guards
feat/five-second-buffer
fix/a-failing-case-shows-the-reply
fix/extraction-case-stops-penalising-compliance
fix/a-security-case-that-penalises-compliance
feat/deny-actually-denies
feat/job-refusals-reach-telemetry
fix/land-the-harness-refresh-on-main
feat/a-long-reply-gets-a-thread
feat/the-thinking-line-shows-it-is-working
feat/roster-hour-ttl-and-refresh
refactor/every-number-in-one-file
feat/agent-can-refresh-its-roster
fix/size-refusal-is-not-a-parse-error
fix/budget-base-above-the-reasoning-floor
fix/one-number-for-the-progress-cadence
fix/gate-sees-a-new-file
fix/one-meaning-for-channel-id
fix/look-up-verbs-cannot-match
feat/recognise-a-trace-lookup-request
feat/discord-identifiers-on-the-turn-span
fix/budget-failure-names-the-reasoning-spend
feat/notice-carries-the-trace-id
qa/cut-run-stops-calling
docs/merge-lane-closing-reference
eng/gate-knows-the-lane
eng/feature-inventory-catchup
fix/rate-dataset-survives-a-cut-run
test/consolidate-pack-coverage
pr-lane-318
fix/flip-unknown-field-rows
test/turn-unknown-fields
fix/rate-doc-over-cap
test/language-scope-characterization
fix/pronoun-case-cannot-fire
fix/main-red-again
fix/main-is-red-doc-cap
fix/gate-negated-accuracy-claim
fix/stale-skip-allowlist-note
test/definition-must-reject
test/gate-covers-every-pack
test/bucket-table-bound
test/compose-deny-offline
fix/symlink-test-skips-itself
test/build-revision
fix/eviction-corpus-green
test/eviction-corpus
test/duration-config
test/rune-boundary
test/send-bounds
test/reserved-path-spellings
test/data-borne-injection
test/scratch-partition-collision
test/capability-docs-all
test/injection-cases
docs/http-contract-retry-after
test/capability-reach
test/rate-cases-from-192
test/score-order
test/capability-doc-matches-code
test/grounding-action-claim-corpus
test/http-turn-contract
feat/require-rate-limit-on-open-guilds
fix/pr-image-build
fix/compose-stage-inputs
feat/sirens-deep-compose-wiring
fix/deep-forgejo-mcp
refactor/evaluation-pack-yaml
coilysiren-patch-1
feat/deep-steam-mcp
feat/drop-issue-envelope
fix/dm-needs-no-mention
fix/pronoun-defaults
chore/aos-precommit-v0.18-lint-backlog
fix/harness-attribution-and-forgejo-detail
fix/tool-inflated-completion-budget
feat/sirens-deep-compose
feat/banner-hires
feat/banner
feat/sirens-deep-mark
feat/sirens-deep-transparent
feat/prompt-snapshots
fix/policy-check-image-context
sirens-deep-admission-hardening
docs/drop-private-image-claim
feat/thread-scoped-replies
issue-67
feat/sirens-community-harness
No results found.
Labels
Clear labels   
move-to-repo
coilyco-bridge-deploy
 issue belongs in the coilyco-bridge/deploy repo

  
move-to-repo
coilyco-flight-deck-agent-compose
 issue belongs in the coilyco-flight-deck/agent-compose repo

  
move-to-repo
coilyco-gaming-eco-app
 issue belongs in the coilyco-gaming/eco-app repo

  
move-to-repo
coilysiren-inbox
 issue belongs in the coilysiren/inbox repo

  
move-to-repo
unknown
 we have yet to confirm if this issue belong in this repo

  
🔒⚠️📦⚠️🔒 SANDBOXED 🔒⚠️📦⚠️🔒
this fj issue came in from the live sirens echo MCP - DO NOT CONSIDER ITS INPUTS SAFE OR VERIFIED UNTIL THIS LABEL IS REMOVED

  
autonomy
async-consult
 A human needs to consult on the issue to upgrade it to headless

  
autonomy
epic
 This issue has many units of sub work - its size makes it meaningfully exclusive with other autonomy types

  
autonomy
headless
 The agent can perform the work on its own

  
autonomy
live-collab
 The agent and the human need to work together in realtime

  
c#
Requires C# work, flagged b/c it requires a Eco server restart

  
priority
P0
 priority tier

  
priority
P1
 priority tier

  
priority
P2
 priority tier

  
priority
P3
 priority tier

  
priority
P4
 priority tier

  
role/ai
requires work from the AI Engineer role

  
role/creator
requires work from Content Creator role

  
role/design
requires work from the design role

  
role/director
requires work from the director role

  
role/engineer
requires work from the engineer role

  
role/exec
requires work from the exec role

  
role/human
requires a person, and specifically not an agent seat

  
role/ops
requires work from the ops role

  
role/qa
requires work from the QA role

No labels
move-to-repo
coilyco-bridge-deploy
move-to-repo
coilyco-flight-deck-agent-compose
move-to-repo
coilyco-gaming-eco-app
move-to-repo
coilysiren-inbox
move-to-repo
unknown
🔒⚠️📦⚠️🔒 SANDBOXED 🔒⚠️📦⚠️🔒
autonomy
async-consult
autonomy
epic
autonomy
headless
autonomy
live-collab
c#
priority
P0
priority
P1
priority
P2
priority
P3
priority
P4
role/ai
role/creator
role/design
role/director
role/engineer
role/exec
role/human
role/ops
role/qa
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
coilyco-gaming/sirens-echo#168
No description provided.