[proxy 05/13] upstream.py: httpx client that injects options.num_ctx (the core fix) #9
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Spec: this implements ONE small step of #2 (the locked reliability-proxy build, aosh leg 04). Do not redesign. Stay tightly scoped to this single step.
Before you start:
git checkout main && git pullso you build on the latest landed step.Fill in
app/upstream.py- this is the highest-value step, the num_ctx injection that kills the 32k truncation.dispatch_chat(logical_name, messages, **opts) -> dict.app.models.get_model. POST with anhttpx.AsyncClientto{backend_url}/api/chat(native ollama API), body:{"model": ollama_tag, "messages": messages, "stream": false, "options": {"num_ctx": <the model's num_ctx>}}.options.num_ctxMUST be set from the registry on every request. Return the parsed JSON.Acceptance: a pytest using
respxor monkeypatched httpx asserts that the outgoing payload to/api/chatcontainsoptions.num_ctx == 49152forfast-think. (Live tower test is gated on ward#330 and handled in step 07 - do NOT block on it here.)When done:
pre-commit run --all-files(or the repo's gate) must pass, commit your work, and push to canonicalmain(this is an ephemeral feature container - finish the merge, do not leave it for a human).docs/FEATURES.mdfromplannedtolandedfor this feature, in the same push.ward agent engineer coilyco-flight-deck/agent-proxy#10 --driver goose🔒 Reserved by
ward agent --driver claude— containerengineer-claude-agent-proxy-9on hostkais-macbook-pro-2.localis carrying this issue (reserved 2026-07-02T16:43:38Z). Concurrentward agentruns are blocked until it finishes or the reservation goes stale (2h0m0s TTL);--forceoverrides.— Claude (she/her), via
ward agentWARD-OUTCOME: done - added a registry-sourced num_ctx test proving fast-think injects options.num_ctx==49152; core injection already landed with the resilience-proxy core.
Honestly this one felt less like building and more like archaeology. I opened the container ready to write
upstream.pyfrom scratch, and found it already fully fleshed out onmain- the num_ctx injection, the caller-can't-override guard, ollama/OpenAI dialect handling, all of it landed earlier as part of the big resilience-proxy-core merge (issue #2), and FEATURES.md already readlanded. So the interesting work was deciding what #9 still means once its deliverable exists.The tempting-but-wrong move was to add the literal
dispatch_chat(logical_name, ...)signature from the spec. But the landed architecture routes throughresilience.dispatch(retry/fallback/circuit-breaker) which callsupstream.chat(backend, model.num_ctx, ...), so a paralleldispatch_chatthat skipped resilience would have been dead code - a redesign the doctrine explicitly forbids. What was genuinely missing versus the acceptance criterion was proof the 49152 comes from the registry for fast-think, not a hand-passed constant. So that's the one thing I added: a tight test that resolves fast-think from the registry and asserts its num_ctx reaches the outgoing /api/chat payload. 38 tests green.Confidence is high - the behavior was already proven end-to-end against the live tower, I just closed the test gap. Two rough edges worth noting: the disk hit 100% mid-run (had to clear the uv cache and drop .venv to even dispatch), and step #10 turned out to be already reserved and in flight by another container, so I did not force the dispatch.