[proxy 06/13] /v1/chat/completions (non-streaming) shaped to OpenAI schema #10

New issue

Closed

opened 2026-06-26 09:48:19 +00:00 by coilysiren · 2 comments

coilysiren commented

2026-06-26 09:48:19 +00:00

Owner

Spec: this implements ONE small step of #2 (the locked reliability-proxy build, aosh leg 04). Do not redesign. Stay tightly scoped to this single step.

Before you start: git checkout main && git pull so you build on the latest landed step.
Add the OpenAI-compatible chat route in app/main.py, calling app.upstream.dispatch_chat.

POST /v1/chat/completions: accept the OpenAI request body (model, messages, ignore extras gracefully). Map model to the logical name, call dispatch_chat, and shape the ollama response into the OpenAI chat.completion schema: {"id", "object":"chat.completion", "model", "choices":[{"index":0, "message":{"role":"assistant","content":...}, "finish_reason":"stop"}], "usage":{"prompt_tokens":prompt_eval_count, "completion_tokens":eval_count, "total_tokens":sum}}.
Non-streaming only here. Streaming comes in step 13.

Acceptance: a pytest with dispatch_chat monkeypatched to return a fake ollama reply asserts the route returns OpenAI-shaped JSON with a choices[0].message.content.

When done:

pre-commit run --all-files (or the repo's gate) must pass, commit your work, and push to canonical main (this is an ephemeral feature container - finish the merge, do not leave it for a human).
Update the matching line in docs/FEATURES.md from planned to landed for this feature, in the same push.
Dispatch the next step: ward agent engineer coilyco-flight-deck/agent-proxy#11 --driver goose

**Spec:** this implements ONE small step of #2 (the locked reliability-proxy build, aosh leg 04). **Do not redesign. Stay tightly scoped to this single step.** **Before you start:** `git checkout main && git pull` so you build on the latest landed step. Add the OpenAI-compatible chat route in `app/main.py`, calling `app.upstream.dispatch_chat`. - `POST /v1/chat/completions`: accept the OpenAI request body (`model`, `messages`, ignore extras gracefully). Map `model` to the logical name, call `dispatch_chat`, and shape the ollama response into the OpenAI `chat.completion` schema: `{"id", "object":"chat.completion", "model", "choices":[{"index":0, "message":{"role":"assistant","content":...}, "finish_reason":"stop"}], "usage":{"prompt_tokens":prompt_eval_count, "completion_tokens":eval_count, "total_tokens":sum}}`. - Non-streaming only here. Streaming comes in step 13. **Acceptance:** a pytest with `dispatch_chat` monkeypatched to return a fake ollama reply asserts the route returns OpenAI-shaped JSON with a `choices[0].message.content`. **When done:** 1. `pre-commit run --all-files` (or the repo's gate) must pass, commit your work, and push to canonical `main` (this is an ephemeral feature container - finish the merge, do not leave it for a human). 2. Update the matching line in `docs/FEATURES.md` from `planned` to `landed` for this feature, in the same push. 3. Dispatch the next step: `ward agent engineer coilyco-flight-deck/agent-proxy#11 --driver goose`

coilyco-ops added the

headless

labels

2026-07-02 16:42:39 +00:00

coilyco-ops commented

2026-07-02 16:44:40 +00:00

Member

🔒 Reserved by ward agent --driver claude — container engineer-claude-agent-proxy-10 on host kais-macbook-pro-2.local is carrying this issue (reserved 2026-07-02T16:44:38Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); --force overrides.

— Claude (she/her), via ward agent

🔒 Reserved by `ward agent --driver claude` — container `engineer-claude-agent-proxy-10` on host `kais-macbook-pro-2.local` is carrying this issue (reserved 2026-07-02T16:44:38Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); `--force` overrides.  — Claude (she/her), via `ward agent`

coilysiren referenced this issue

2026-07-02 16:47:00 +00:00

[proxy 05/13] upstream.py: httpx client that injects options.num_ctx (the core fix) #9

coilysiren referenced this issue from a commit

2026-07-02 16:48:31 +00:00

test: harden /v1/chat/completions acceptance to the full OpenAI schema

coilysiren closed this issue

2026-07-02 16:48:31 +00:00

coilysiren commented

2026-07-02 16:49:18 +00:00

Author

Owner

WARD-OUTCOME: done - non-streaming /v1/chat/completions OpenAI shaping was already landed; hardened its acceptance test to the full schema and pushed to main.

Honest retrospective: this step turned out to be already-built. The non-streaming route, the exact chat.completion shaping, and a faked-dispatch acceptance test all landed earlier with the resilience-proxy core (#2), and the FEATURES line already read landed. The issue references app.upstream.dispatch_chat, but the code evolved past that name - dispatch runs through the queue + resilience layer into upstream.chat. Rather than redesign backward to invent a dispatch_chat shim (which the container doctrine forbids and which would regress the queue path), I stayed scoped: I tightened the existing acceptance test to assert the whole schema this step locks in - assistant role, model echo, choice index, and a usage block where total_tokens == prompt + completion.

What fought back: nothing in the code, but the canonical main moved under me twice mid-push (other proxy-step containers landing #9, models/analysis API work), so I fetched-merged-retested-pushed a couple of times. Clean merges, 52 tests green after each.

Confidence is high in the result - the contract is now more strictly pinned and everything's green. Two rough edges worth flagging: (1) the next-step dispatch (ward agent engineer ...#11) could not run - this daemonless feature container has no Docker socket, so create sibling failed; a human or the host loop needs to kick off #11. (2) These granular proxy-step issues are all shadow-boxing an already-complete build, so each is really a verify-and-close pass, not fresh implementation - worth knowing for the rest of the 13.

WARD-OUTCOME: done - non-streaming /v1/chat/completions OpenAI shaping was already landed; hardened its acceptance test to the full schema and pushed to main. Honest retrospective: this step turned out to be already-built. The non-streaming route, the exact chat.completion shaping, and a faked-dispatch acceptance test all landed earlier with the resilience-proxy core (#2), and the FEATURES line already read landed. The issue references `app.upstream.dispatch_chat`, but the code evolved past that name - dispatch runs through the queue + resilience layer into `upstream.chat`. Rather than redesign backward to invent a `dispatch_chat` shim (which the container doctrine forbids and which would regress the queue path), I stayed scoped: I tightened the existing acceptance test to assert the whole schema this step locks in - assistant role, model echo, choice index, and a usage block where total_tokens == prompt + completion. What fought back: nothing in the code, but the canonical main moved under me twice mid-push (other proxy-step containers landing #9, models/analysis API work), so I fetched-merged-retested-pushed a couple of times. Clean merges, 52 tests green after each. Confidence is high in the result - the contract is now more strictly pinned and everything's green. Two rough edges worth flagging: (1) the next-step dispatch (`ward agent engineer ...#11`) could not run - this daemonless feature container has no Docker socket, so `create sibling` failed; a human or the host loop needs to kick off #11. (2) These granular proxy-step issues are all shadow-boxing an already-complete build, so each is really a verify-and-close pass, not fresh implementation - worth knowing for the rest of the 13.