Retire the logical-model aliases: pass-through tags + auto num_ctx #32
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Retire the logical-model aliases: pass-through tags + auto num_ctx
Director direction from Kai: retire the
fast-think/fast/ctx-think/ctx/tunelogical-model alias table. This issue is the interim, phase-1simplification. The larger litellm-as-core re-core that supersedes the routing
layer entirely is tracked in
coilyco-bridge/agentic-os-hardware#25and iscompatible with this change.
Why
app/models.pyis hand-maintained and drifts from whatis actually pulled on a backend. Its tags (
qwen3:30b-a3b,qwen3-coder:30b,qwen3:32b) are assumptions, and itsnum_ctxvalues (49152 / 32768) areguesses standing in until a benchmark that has not happened.
/api/tagsreturns
details.context_lengthper model (confirmed live on the Windowstower:
qwen3:8b= 40960,qwen3:4b= 262144), and/api/showgives more.The proxy can derive a safe
num_ctxfrom the truth instead of a static table.Change
qwen3:4b) asmodel. No logicalindirection.
context_lengthonce(cached), and injects `options.num_ctx = min(context_length, configured_ceiling)
. **The caller still can never overridenum_ctx`** - that invariantis the whole wedge and must stay.
/v1/modelslists the tags actually present on the backend (from/api/tags), not a static alias list.(
analysis.py), response validation, retry/backoff, fallback chain, and theper-backend circuit breaker. Only the logical-alias indirection in
models.pyand its
/v1/modelslisting go.Backward-compat decision for the engineer
No production harness is repointed yet (M4 is not done), so a hard cut is
defensible. If a thin shim is cheaper to land safely, gate it behind
PROXY_ALIAS_COMPATdefaulting off and note the removal milestone. Pick oneand say which in the PR/commit.
Acceptance
num_ctxderivation, mocking/api/tags(
qwen3:8b-> 40960,qwen3:4b-> 262144) and asserting the injected value andthe caller-cannot-override invariant still hold.
/v1/modelsreflects live backend tags.ward exec test),ward exec boot-probeandward exec smokegreen.docs/FEATURES.md,docs/proxy.md, anddocs/ROADMAP.mdupdated where theydescribe the logical-alias table. Note aosh#25 as the superseding design track.
repointing.
Local validation available
On the Windows tower this runs against live ollama at
http://host.docker.internal:11434(Docker Desktop host gateway) withqwen3:4b(ctx 262144) and
qwen3:8b(ctx 40960). A dispatched container reaches it thesame way.
Filed from the agent-proxy read-only director surface.
🔒 Reserved by
ward agent --driver claude— containerengineer-claude-agent-proxy-32on hostKAI-DESKTOP-TOWERis carrying this issue (reserved 2026-07-03T18:26:15Z). Concurrentward agentruns are blocked until it finishes or the reservation goes stale (2h0m0s TTL);--forceoverrides.Do not comment on or edit this issue to steer the run while it is reserved. The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a new issue, dispatched fresh — that is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).
— Claude (she/her), via
ward agentWARD-OUTCOME: done - pass-through ollama tags + auto num_ctx from /api/tags landed on main (
8655304), alias table retired, closes #32.Retro: this one was cleaner than I expected going in. The dispatch engine was already shaped around a
(name, num_ctx, backends)record, so retiring the alias table mostly meant swapping the static registry for an async/api/tagscatalog + aderive_num_ctxand leaving resilience/queue/upstream untouched. I did a hard cut (noPROXY_ALIAS_COMPATshim) since M4 hasn't repointed any harness — cheaper and honest.What fought back a little: two robustness forks the acceptance criteria didn't dictate but that mattered. (1) fail-open when the catalog fetch fails, so a transient tower outage surfaces as a 502 from dispatch instead of a false 404 on a real tag — and only caching successful reads so a tower that comes up after boot is still discovered. (2)
/v1/modelsis now backend-derived, so a tower-less boot-probe correctly returns an empty list; I had to relaxprobe_endpoints.shfrom 'has a model id' to 'is list-shaped', which is the right contract now. The merge with main also landed the new reliability-harness work, which carried a few stalefast-thinkreferences I swept.Confidence: high on the offline surface — 68 tests, boot-probe, and smoke all green, and the caller-cannot-override invariant is double-guarded (OpenAI translation drops num_ctx, upstream forces it). The one thing genuinely unverified is the live
details.context_lengthshape from the tower's/api/tags; I trusted the issue's stated contract (qwen3:8b=40960, qwen3:4b=262144) and mocked to it, but no container run hit real ollama. Worth a quick liveward exec proofagainst the tower before leaning on it.Rough edges / follow-ups worth filing: the
logical_modelmetric label andLogicalModelclass name are kept (consistency with the observability surface), so 'logical' lingers in the vocabulary even though the indirection is gone — a rename could come with the aosh#25 litellm re-core rather than churn dashboards now. And the catalog cache has no TTL/refresh — fine for phase 1, but a model pull won't show in/v1/modelsuntil a restart.