Retire the logical-model aliases: pass-through tags + auto num_ctx #32

Closed
opened 2026-07-03 18:24:57 +00:00 by coilysiren · 2 comments
Owner

Retire the logical-model aliases: pass-through tags + auto num_ctx

Director direction from Kai: retire the fast-think / fast / ctx-think /
ctx / tune logical-model alias table.
This issue is the interim, phase-1
simplification. The larger litellm-as-core re-core that supersedes the routing
layer entirely is tracked in coilyco-bridge/agentic-os-hardware#25 and is
compatible with this change.

Why

  • The alias table in app/models.py is hand-maintained and drifts from what
    is actually pulled on a backend. Its tags (qwen3:30b-a3b, qwen3-coder:30b,
    qwen3:32b) are assumptions, and its num_ctx values (49152 / 32768) are
    guesses standing in until a benchmark that has not happened.
  • Ollama already reports each model's real context window: /api/tags
    returns details.context_length per model (confirmed live on the Windows
    tower: qwen3:8b = 40960, qwen3:4b = 262144), and /api/show gives more.
    The proxy can derive a safe num_ctx from the truth instead of a static table.

Change

  • Harnesses pass the real ollama tag (e.g. qwen3:4b) as model. No logical
    indirection.
  • The proxy resolves the backend, reads that model's context_length once
    (cached), and injects `options.num_ctx = min(context_length, configured_ceiling)
    • headroom. **The caller still can never override num_ctx`** - that invariant
      is the whole wedge and must stay.
  • /v1/models lists the tags actually present on the backend (from
    /api/tags), not a static alias list.
  • Keep everything else: num_ctx injection, context-budget guard
    (analysis.py), response validation, retry/backoff, fallback chain, and the
    per-backend circuit breaker. Only the logical-alias indirection in models.py
    and its /v1/models listing go.

Backward-compat decision for the engineer

No production harness is repointed yet (M4 is not done), so a hard cut is
defensible. If a thin shim is cheaper to land safely, gate it behind
PROXY_ALIAS_COMPAT defaulting off and note the removal milestone. Pick one
and say which in the PR/commit.

Acceptance

  • Unit tests for the num_ctx derivation, mocking /api/tags
    (qwen3:8b -> 40960, qwen3:4b -> 262144) and asserting the injected value and
    the caller-cannot-override invariant still hold.
  • /v1/models reflects live backend tags.
  • Offline suite green (ward exec test), ward exec boot-probe and
    ward exec smoke green.
  • docs/FEATURES.md, docs/proxy.md, and docs/ROADMAP.md updated where they
    describe the logical-alias table. Note aosh#25 as the superseding design track.
  • Stays inside phase 1. No litellm adoption here, no deploy manifests, no harness
    repointing.

Local validation available

On the Windows tower this runs against live ollama at
http://host.docker.internal:11434 (Docker Desktop host gateway) with qwen3:4b
(ctx 262144) and qwen3:8b (ctx 40960). A dispatched container reaches it the
same way.

Filed from the agent-proxy read-only director surface.

## Retire the logical-model aliases: pass-through tags + auto num_ctx Director direction from Kai: **retire the `fast-think` / `fast` / `ctx-think` / `ctx` / `tune` logical-model alias table.** This issue is the interim, phase-1 simplification. The larger litellm-as-core re-core that supersedes the routing layer entirely is tracked in `coilyco-bridge/agentic-os-hardware#25` and is compatible with this change. **Why** - The alias table in `app/models.py` is hand-maintained and **drifts** from what is actually pulled on a backend. Its tags (`qwen3:30b-a3b`, `qwen3-coder:30b`, `qwen3:32b`) are assumptions, and its `num_ctx` values (49152 / 32768) are guesses standing in until a benchmark that has not happened. - Ollama **already reports each model's real context window**: `/api/tags` returns `details.context_length` per model (confirmed live on the Windows tower: `qwen3:8b` = 40960, `qwen3:4b` = 262144), and `/api/show` gives more. The proxy can derive a safe `num_ctx` from the truth instead of a static table. **Change** - Harnesses pass the **real ollama tag** (e.g. `qwen3:4b`) as `model`. No logical indirection. - The proxy resolves the backend, reads that model's `context_length` once (cached), and injects `options.num_ctx = min(context_length, configured_ceiling) - headroom`. **The caller still can never override `num_ctx`** - that invariant is the whole wedge and must stay. - `/v1/models` lists the tags **actually present** on the backend (from `/api/tags`), not a static alias list. - **Keep everything else**: num_ctx injection, context-budget guard (`analysis.py`), response validation, retry/backoff, fallback chain, and the per-backend circuit breaker. Only the logical-alias indirection in `models.py` and its `/v1/models` listing go. **Backward-compat decision for the engineer** No production harness is repointed yet (M4 is not done), so a **hard cut** is defensible. If a thin shim is cheaper to land safely, gate it behind `PROXY_ALIAS_COMPAT` defaulting **off** and note the removal milestone. Pick one and say which in the PR/commit. **Acceptance** - Unit tests for the `num_ctx` derivation, mocking `/api/tags` (`qwen3:8b` -> 40960, `qwen3:4b` -> 262144) and asserting the injected value and the **caller-cannot-override** invariant still hold. - `/v1/models` reflects live backend tags. - Offline suite green (`ward exec test`), `ward exec boot-probe` and `ward exec smoke` green. - `docs/FEATURES.md`, `docs/proxy.md`, and `docs/ROADMAP.md` updated where they describe the logical-alias table. Note aosh#25 as the superseding design track. - Stays inside phase 1. No litellm adoption here, no deploy manifests, no harness repointing. **Local validation available** On the Windows tower this runs against live ollama at `http://host.docker.internal:11434` (Docker Desktop host gateway) with `qwen3:4b` (ctx 262144) and `qwen3:8b` (ctx 40960). A dispatched container reaches it the same way. Filed from the agent-proxy read-only director surface.
Member

🔒 Reserved by ward agent --driver claude — container engineer-claude-agent-proxy-32 on host KAI-DESKTOP-TOWER is carrying this issue (reserved 2026-07-03T18:26:15Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); --force overrides.

Do not comment on or edit this issue to steer the run while it is reserved. The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a new issue, dispatched fresh — that is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).

— Claude (she/her), via ward agent

<!-- ward-agent-reservation --> 🔒 Reserved by `ward agent --driver claude` — container `engineer-claude-agent-proxy-32` on host `KAI-DESKTOP-TOWER` is carrying this issue (reserved 2026-07-03T18:26:15Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); `--force` overrides. **Do not comment on or edit this issue to steer the run while it is reserved.** The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a **new issue, dispatched fresh** — that is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494). <!-- ward-agent-signature --> — Claude (she/her), via `ward agent`
Author
Owner

WARD-OUTCOME: done - pass-through ollama tags + auto num_ctx from /api/tags landed on main (8655304), alias table retired, closes #32.

Retro: this one was cleaner than I expected going in. The dispatch engine was already shaped around a (name, num_ctx, backends) record, so retiring the alias table mostly meant swapping the static registry for an async /api/tags catalog + a derive_num_ctx and leaving resilience/queue/upstream untouched. I did a hard cut (no PROXY_ALIAS_COMPAT shim) since M4 hasn't repointed any harness — cheaper and honest.

What fought back a little: two robustness forks the acceptance criteria didn't dictate but that mattered. (1) fail-open when the catalog fetch fails, so a transient tower outage surfaces as a 502 from dispatch instead of a false 404 on a real tag — and only caching successful reads so a tower that comes up after boot is still discovered. (2) /v1/models is now backend-derived, so a tower-less boot-probe correctly returns an empty list; I had to relax probe_endpoints.sh from 'has a model id' to 'is list-shaped', which is the right contract now. The merge with main also landed the new reliability-harness work, which carried a few stale fast-think references I swept.

Confidence: high on the offline surface — 68 tests, boot-probe, and smoke all green, and the caller-cannot-override invariant is double-guarded (OpenAI translation drops num_ctx, upstream forces it). The one thing genuinely unverified is the live details.context_length shape from the tower's /api/tags; I trusted the issue's stated contract (qwen3:8b=40960, qwen3:4b=262144) and mocked to it, but no container run hit real ollama. Worth a quick live ward exec proof against the tower before leaning on it.

Rough edges / follow-ups worth filing: the logical_model metric label and LogicalModel class name are kept (consistency with the observability surface), so 'logical' lingers in the vocabulary even though the indirection is gone — a rename could come with the aosh#25 litellm re-core rather than churn dashboards now. And the catalog cache has no TTL/refresh — fine for phase 1, but a model pull won't show in /v1/models until a restart.

WARD-OUTCOME: done - pass-through ollama tags + auto num_ctx from /api/tags landed on main (8655304), alias table retired, closes #32. Retro: this one was cleaner than I expected going in. The dispatch engine was already shaped around a `(name, num_ctx, backends)` record, so retiring the alias *table* mostly meant swapping the static registry for an async `/api/tags` catalog + a `derive_num_ctx` and leaving resilience/queue/upstream untouched. I did a hard cut (no `PROXY_ALIAS_COMPAT` shim) since M4 hasn't repointed any harness — cheaper and honest. What fought back a little: two robustness forks the acceptance criteria didn't dictate but that mattered. (1) fail-*open* when the catalog fetch fails, so a transient tower outage surfaces as a 502 from dispatch instead of a false 404 on a real tag — and only caching *successful* reads so a tower that comes up after boot is still discovered. (2) `/v1/models` is now backend-derived, so a tower-less boot-probe correctly returns an empty list; I had to relax `probe_endpoints.sh` from 'has a model id' to 'is list-shaped', which is the right contract now. The merge with main also landed the new reliability-harness work, which carried a few stale `fast-think` references I swept. Confidence: high on the offline surface — 68 tests, boot-probe, and smoke all green, and the caller-cannot-override invariant is double-guarded (OpenAI translation drops num_ctx, upstream forces it). The one thing genuinely unverified is the *live* `details.context_length` shape from the tower's `/api/tags`; I trusted the issue's stated contract (qwen3:8b=40960, qwen3:4b=262144) and mocked to it, but no container run hit real ollama. Worth a quick live `ward exec proof` against the tower before leaning on it. Rough edges / follow-ups worth filing: the `logical_model` metric label and `LogicalModel` class name are kept (consistency with the observability surface), so 'logical' lingers in the vocabulary even though the indirection is gone — a rename could come with the aosh#25 litellm re-core rather than churn dashboards now. And the catalog cache has no TTL/refresh — fine for phase 1, but a model pull won't show in `/v1/models` until a restart.
Sign in to join this conversation.
No description provided.