[proxy 04/13] models.py: logical-model registry with per-model num_ctx #8

New issue

Closed

opened 2026-06-26 09:48:18 +00:00 by coilysiren · 2 comments

coilysiren commented

2026-06-26 09:48:18 +00:00

Owner

Spec: this implements ONE small step of #2 (the locked reliability-proxy build, aosh leg 04). Do not redesign. Stay tightly scoped to this single step.

Before you start: git checkout main && git pull so you build on the latest landed step.
Fill in app/models.py - the backend registry. This is data, no I/O.

A dict LOGICAL_MODELS mapping logical name -> {"backend_url": str, "ollama_tag": str, "num_ctx": int}.
Seed these logical names (ollama_tag is a placeholder you can set to the name; backend_url defaults to settings.ollama_base_url):
- fast-think -> num_ctx 49152 (Qwen3-A3B, proven safe in leg 01)
- fast -> 32768
- ctx-think -> 32768
- ctx -> 32768
- tune -> 32768
Export get_model(name) -> dict that raises KeyError/returns None for unknown names, and list_models() -> list[str].

Acceptance: uv run python -c "from app.models import get_model; print(get_model('fast-think')['num_ctx'])" prints 49152. Add a pytest.

When done:

pre-commit run --all-files (or the repo's gate) must pass, commit your work, and push to canonical main (this is an ephemeral feature container - finish the merge, do not leave it for a human).
Update the matching line in docs/FEATURES.md from planned to landed for this feature, in the same push.
Dispatch the next step: ward agent engineer coilyco-flight-deck/agent-proxy#9 --driver goose

**Spec:** this implements ONE small step of #2 (the locked reliability-proxy build, aosh leg 04). **Do not redesign. Stay tightly scoped to this single step.** **Before you start:** `git checkout main && git pull` so you build on the latest landed step. Fill in `app/models.py` - the backend registry. **This is data, no I/O.** - A dict `LOGICAL_MODELS` mapping logical name -> `{"backend_url": str, "ollama_tag": str, "num_ctx": int}`. - Seed these logical names (ollama_tag is a placeholder you can set to the name; backend_url defaults to `settings.ollama_base_url`): - `fast-think` -> num_ctx **49152** (Qwen3-A3B, proven safe in leg 01) - `fast` -> 32768 - `ctx-think` -> 32768 - `ctx` -> 32768 - `tune` -> 32768 - Export `get_model(name) -> dict` that raises `KeyError`/returns None for unknown names, and `list_models() -> list[str]`. **Acceptance:** `uv run python -c "from app.models import get_model; print(get_model('fast-think')['num_ctx'])"` prints 49152. Add a pytest. **When done:** 1. `pre-commit run --all-files` (or the repo's gate) must pass, commit your work, and push to canonical `main` (this is an ephemeral feature container - finish the merge, do not leave it for a human). 2. Update the matching line in `docs/FEATURES.md` from `planned` to `landed` for this feature, in the same push. 3. Dispatch the next step: `ward agent engineer coilyco-flight-deck/agent-proxy#9 --driver goose`

coilyco-ops added the

headless

labels

2026-07-02 16:42:37 +00:00

coilyco-ops commented

2026-07-02 16:43:34 +00:00

Member

🔒 Reserved by ward agent --driver claude — container engineer-claude-agent-proxy-8 on host kais-macbook-pro-2.local is carrying this issue (reserved 2026-07-02T16:43:32Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); --force overrides.

— Claude (she/her), via ward agent

🔒 Reserved by `ward agent --driver claude` — container `engineer-claude-agent-proxy-8` on host `kais-macbook-pro-2.local` is carrying this issue (reserved 2026-07-02T16:43:32Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); `--force` overrides.  — Claude (she/her), via `ward agent`

coilysiren referenced this issue

2026-07-02 16:46:51 +00:00

[proxy 03/13] main.py: FastAPI app with /healthz and /metrics #7

coilysiren referenced this issue from a commit

2026-07-02 16:47:45 +00:00

feat(models): flat-dict logical-model API (get_model/list_models/LOGICAL_MODELS)

coilysiren closed this issue

2026-07-02 16:47:45 +00:00

coilysiren commented

2026-07-02 16:48:16 +00:00

Author

Owner

WARD-OUTCOME: done - added the flat-dict get_model/list_models/LOGICAL_MODELS API over the existing registry; landed on main as 4b6d962.

The surprise here was that app/models.py had already evolved well past this step's spec - a full Registry/LogicalModel design with fallback chains and dialect adapters was landed, but the small flat {backend_url, ollama_tag, num_ctx} contract this step actually names (and its acceptance test) simply didn't exist yet. So rather than write the naive dict the issue literally describes, I added get_model/list_models/LOGICAL_MODELS as a thin facade over the registry's primary backend. That keeps the richer design intact and still makes the acceptance (get_model('fast-think')['num_ctx'] == 49152) pass. Felt like the right call under 'don't redesign', but it's a judgement call worth a second look.

Two things fought back: the box's backing disk was 100% full, so the first commit died with ENOSPC (retried fine), and main moved under me twice from concurrent sibling runs - rebased cleanly both times. Confident in the result: 50 tests green including a new test_models.py. The next step (#9) was already reserved and in fact already landed by another run, so my dispatch correctly declined. One rough edge to flag: default black wants to reflow pre-existing lines in models.py (the repo clearly uses a wider line length than black's 88 default and ships no black config), so the formatter story is a little ambiguous - ruff is the real gate and it passes. Might be worth pinning a [tool.black] line-length to match ruff.

WARD-OUTCOME: done - added the flat-dict `get_model`/`list_models`/`LOGICAL_MODELS` API over the existing registry; landed on main as 4b6d962. The surprise here was that `app/models.py` had already evolved well past this step's spec - a full `Registry`/`LogicalModel` design with fallback chains and dialect adapters was landed, but the small flat `{backend_url, ollama_tag, num_ctx}` contract this step actually names (and its acceptance test) simply didn't exist yet. So rather than write the naive dict the issue literally describes, I added `get_model`/`list_models`/`LOGICAL_MODELS` as a thin facade over the registry's primary backend. That keeps the richer design intact and still makes the acceptance (`get_model('fast-think')['num_ctx'] == 49152`) pass. Felt like the right call under 'don't redesign', but it's a judgement call worth a second look. Two things fought back: the box's backing disk was 100% full, so the first commit died with ENOSPC (retried fine), and `main` moved under me twice from concurrent sibling runs - rebased cleanly both times. Confident in the result: 50 tests green including a new `test_models.py`. The next step (#9) was already reserved and in fact already landed by another run, so my dispatch correctly declined. One rough edge to flag: default `black` wants to reflow pre-existing lines in `models.py` (the repo clearly uses a wider line length than black's 88 default and ships no black config), so the formatter story is a little ambiguous - `ruff` is the real gate and it passes. Might be worth pinning a `[tool.black]` line-length to match ruff.