[proxy 13/13] OpenAI surface polish: streaming chat, /v1/completions, /v1/models

coilysiren commented

2026-06-26 09:48:20 +00:00

Owner

Spec: this implements ONE small step of #2 (the locked reliability-proxy build, aosh leg 04). Do not redesign. Stay tightly scoped to this single step.

Before you start: git checkout main && git pull so you build on the latest landed step.
Complete the OpenAI-compatible surface in app/main.py + app/upstream.py.

Streaming POST /v1/chat/completions when stream: true: stream ollama /api/chat (stream=true) and re-emit as OpenAI SSE chat.completion.chunk events ending with data: [DONE].
POST /v1/completions: legacy text-completion shape, internally mapped to a single-user-message chat call.
GET /v1/models: list the logical names from app.models.list_models() in OpenAI {"object":"list","data":[{"id":name,"object":"model"}...]} shape.

Acceptance: pytests: /v1/models lists fast-think; a streaming request yields SSE chunks then [DONE]; /v1/completions returns a completion. This closes the core build of #2 - leave a comment on #2 noting the surface is complete.

When done:

pre-commit run --all-files (or the repo's gate) must pass, commit your work, and push to canonical main (this is an ephemeral feature container - finish the merge, do not leave it for a human).
Update the matching line in docs/FEATURES.md from planned to landed for this feature, in the same push.
This is the final build step. After pushing, comment on #2 that the phase-1 reliability proxy core is built and locally testable, and (if the tower route ward#330 is up) that the live num_ctx proof passes. No further dispatch needed.

**Spec:** this implements ONE small step of #2 (the locked reliability-proxy build, aosh leg 04). **Do not redesign. Stay tightly scoped to this single step.** **Before you start:** `git checkout main && git pull` so you build on the latest landed step. Complete the OpenAI-compatible surface in `app/main.py` + `app/upstream.py`. - Streaming `POST /v1/chat/completions` when `stream: true`: stream ollama `/api/chat` (stream=true) and re-emit as OpenAI SSE `chat.completion.chunk` events ending with `data: [DONE]`. - `POST /v1/completions`: legacy text-completion shape, internally mapped to a single-user-message chat call. - `GET /v1/models`: list the logical names from `app.models.list_models()` in OpenAI `{"object":"list","data":[{"id":name,"object":"model"}...]}` shape. **Acceptance:** pytests: /v1/models lists `fast-think`; a streaming request yields SSE chunks then [DONE]; /v1/completions returns a completion. This closes the core build of #2 - leave a comment on #2 noting the surface is complete. **When done:** 1. `pre-commit run --all-files` (or the repo's gate) must pass, commit your work, and push to canonical `main` (this is an ephemeral feature container - finish the merge, do not leave it for a human). 2. Update the matching line in `docs/FEATURES.md` from `planned` to `landed` for this feature, in the same push. 3. This is the final build step. After pushing, comment on #2 that the phase-1 reliability proxy core is built and locally testable, and (if the tower route ward#330 is up) that the live num_ctx proof passes. No further dispatch needed.

coilyco-ops added the

headless

P4

labels

2026-07-02 16:42:44 +00:00

coilyco-ops commented

2026-07-02 16:45:59 +00:00

Member

🔒 Reserved by ward agent --driver claude — container engineer-claude-agent-proxy-17 on host kais-macbook-pro-2.local is carrying this issue (reserved 2026-07-02T16:45:56Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); --force overrides.

— Claude (she/her), via ward agent

🔒 Reserved by `ward agent --driver claude` — container `engineer-claude-agent-proxy-17` on host `kais-macbook-pro-2.local` is carrying this issue (reserved 2026-07-02T16:45:56Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); `--force` overrides.  — Claude (she/her), via `ward agent`

coilysiren commented

2026-07-02 16:47:35 +00:00

Author

Owner

🔓 Reservation released by ward container reap — container agent-proxy (--driver claude) exited without launching the agent (smoke-test death, ward#222/#264), so the hold it took is retracted. A plain ward agent retry no longer needs --force.

— Claude (she/her), via ward agent

🔓 Reservation released by `ward container reap` — container `agent-proxy` (`--driver claude`) exited without launching the agent (smoke-test death, ward#222/#264), so the hold it took is retracted. A plain `ward agent` retry no longer needs `--force`.  — Claude (she/her), via `ward agent`

coilysiren commented

2026-07-02 16:48:30 +00:00

Author

Owner

🔒 Reserved by ward agent --driver goose — container engineer-goose-agent-proxy-17 on host 1afbcb384a27 is carrying this issue (reserved 2026-07-02T16:48:30Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); --force overrides.

— Goose, via ward agent

🔒 Reserved by `ward agent --driver goose` — container `engineer-goose-agent-proxy-17` on host `1afbcb384a27` is carrying this issue (reserved 2026-07-02T16:48:30Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); `--force` overrides.  — Goose, via `ward agent`

coilysiren referenced this issue

2026-07-02 16:49:06 +00:00

[proxy 12/13] analysis.py: context-budget guard (token count + trim) #16

coilyco-ops commented

2026-07-02 17:47:42 +00:00

Member

🔒 Reserved by ward agent --driver claude — container engineer-claude-agent-proxy-17 on host kais-macbook-pro-2.local is carrying this issue (reserved 2026-07-02T17:47:41Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); --force overrides.

The pre-flight judged this issue GO for an unattended run. Its justification:

pre-flight read (GO)

Context to front-load: Before the first edit in the clone I will read (1) app/main.py — the existing route definitions, how the non-streaming /v1/chat/completions is wired, the FastAPI/response conventions and error shapes already in use; (2) app/upstream.py — how ollama /api/chat is currently called (client, base URL, auth, timeout), so the streaming stream=true path and the /v1/completions→chat mapping reuse that plumbing rather than reinventing it; (3) app/models.py for the exact list_models() signature and what fast-think resolves to; (4) the existing tests/ for the pytest fixtures/style so the three new acceptance tests match convention; (5) docs/FEATURES.md to find the exact planned line to flip; and (6) the repo gate (pre-commit/ward verb) plus #2's comment expectations. I will actually open each of these in the fresh clone before editing, not defer them.

Main risk: the precise ollama streaming chunk schema and how cleanly the current upstream.py supports a streaming generator (async vs sync httpx, whether SSE re-emission needs a StreamingResponse helper that may not yet exist) — plus whether fast-think and list_models() are named exactly as the issue assumes. All of that is readable in the clone and fully reversible, the scope is tight and test-anchored, and the only cross-issue action is a comment on #2 (non-destructive). The reservation-release comment is just container-reap bookkeeping and does not change the spec.

GO

— Claude (she/her), via ward agent

🔒 Reserved by `ward agent --driver claude` — container `engineer-claude-agent-proxy-17` on host `kais-macbook-pro-2.local` is carrying this issue (reserved 2026-07-02T17:47:41Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); `--force` overrides. The pre-flight judged this issue **GO** for an unattended run. Its justification: <details><summary>pre-flight read (GO)</summary> **Context to front-load:** Before the first edit in the clone I will read (1) `app/main.py` — the existing route definitions, how the non-streaming `/v1/chat/completions` is wired, the FastAPI/response conventions and error shapes already in use; (2) `app/upstream.py` — how ollama `/api/chat` is currently called (client, base URL, auth, timeout), so the streaming `stream=true` path and the `/v1/completions`→chat mapping reuse that plumbing rather than reinventing it; (3) `app/models.py` for the exact `list_models()` signature and what `fast-think` resolves to; (4) the existing `tests/` for the pytest fixtures/style so the three new acceptance tests match convention; (5) `docs/FEATURES.md` to find the exact `planned` line to flip; and (6) the repo gate (`pre-commit`/ward verb) plus `#2`'s comment expectations. I will actually open each of these in the fresh clone before editing, not defer them. Main risk: the precise ollama streaming chunk schema and how cleanly the current `upstream.py` supports a streaming generator (async vs sync httpx, whether SSE re-emission needs a `StreamingResponse` helper that may not yet exist) — plus whether `fast-think` and `list_models()` are named exactly as the issue assumes. All of that is readable in the clone and fully reversible, the scope is tight and test-anchored, and the only cross-issue action is a comment on #2 (non-destructive). The reservation-release comment is just container-reap bookkeeping and does not change the spec. GO </details>  — Claude (she/her), via `ward agent`

coilysiren commented

2026-07-02 17:47:51 +00:00

Author

Owner

🔓 Reservation released by ward container reap — container agent-proxy (--driver claude) exited without launching the agent (smoke-test death, ward#222/#264), so the hold it took is retracted. A plain ward agent retry no longer needs --force.

— Claude (she/her), via ward agent

🔓 Reservation released by `ward container reap` — container `agent-proxy` (`--driver claude`) exited without launching the agent (smoke-test death, ward#222/#264), so the hold it took is retracted. A plain `ward agent` retry no longer needs `--force`.  — Claude (she/her), via `ward agent`

coilysiren commented

2026-07-03 19:33:31 +00:00

Author

Owner

Landed. app/main.py serves streaming and non-streaming /v1/chat/completions, /v1/completions, and /v1/models, covered by tests/test_api.py. Note: /v1/models now lists live ollama tags rather than the logical name fast-think after the alias retirement in #32. Closing as done.

coilysiren closed this issue

2026-07-03 19:33:31 +00:00

coilysiren referenced this issue from coilyco-flight-deck/ward

2026-07-14 15:27:09 +00:00

director should emit colorized, space buffered, visually distinct messages about the categories of actions its performing on startup #1283

Rows
Columns

[proxy 13/13] OpenAI surface polish: streaming chat, /v1/completions, /v1/models #17