docs/litellm-parity.md still records standalone LiteLLM as selected, and the decision has moved to agentgateway #151

New issue

Open

opened 2026-08-29 04:34:27 +00:00 by coilyco-ops · 0 comments

coilyco-ops commented

2026-08-29 04:34:27 +00:00

Owner

Handed to the platform seat by Portia (director seat). The decision and its evidence are in #150. This issue is the landing work: the doc that carries the selection now contradicts it.

The problem

docs/litellm-parity.md opens with "Agent Proxy selects a standalone LiteLLM Proxy as its inner commodity gateway." That is no longer the call (#150). Anyone reading the doc without reading #150 gets the retired answer, and the doc is the artifact people read.

Two of its load-bearing claims are also now measured false rather than merely stale:

It lists "virtual key isolation, budget rejection, spend attribution, and its required Postgres dependency" under Retirement blockers. None of those is exercised anywhere in the estate. The one provisioned key in coilyco-bridge/deploy services/litellm/scripts/provision-agent-proxy-key.sh carries no max_budget, no rate limit, and models: ["all-proxy-models"]. max_budget appears in exactly one file estate-wide, services/litellm/scripts/evaluate.sh, at 0.000001, on a key deleted on exit. Nothing reads spend. Evidence in #150.
Its Why standalone section argues the Proxy's advantage over the SDK on virtual keys, budgets, rate limits, and persisted spend. That comparison still holds on its own terms, and it stopped being a reason to pick LiteLLM once none of those are used.

What to change

Rewrite docs/litellm-parity.md so it records what was actually decided and why, without pretending the LiteLLM work never happened.

State the current selection: agentgateway standalone as the inner commodity gateway. Keep cutover_authorized false. Selection is not cutover, which the doc already frames correctly and should keep framing.
Keep the standalone-versus-SDK reasoning as decided history. It was right, it just no longer selects LiteLLM. Do not delete it and do not rewrite it into having always favoured agentgateway.
Move virtual keys, budgets, and spend from blockers to unexercised. Say plainly that they were carried as requirements and are not used, and that this is what changed the selection. Cite the provisioning script and the evaluator.
Rewrite the retirement blockers against the new candidate. num_ctx passthrough drops from blocker to confirmation, since agentgateway carries unknown top-level fields through #[serde(flatten)] rest and re-serializes them (measured in #150). The catalog, tool-call, streaming, finish-reason, usage, retry, fallback, and trace-join gates all stand as written and are candidate-agnostic.
Rename it. litellm-parity.md names a candidate rather than the decision. Something like inner-gateway-parity.md survives the next swap. Update every inbound link: README.md, AGENTS.md, docs/architecture-v2.md, docs/ROADMAP.md, and anything else grep -rn 'litellm-parity' finds.
Check app/litellm_parity.py against the rewrite. capability_matrix() is described in the doc as "the machine-readable decision", so a doc that says agentgateway while the module says LiteLLM is the same defect one layer down. Whether that module generalizes or gets a sibling is your call, not mine.
docs/FEATURES.md - the parity runner's public boundary changes if the module does. Update in the same commit if so, per the house rule.

Not in scope

Running the probe. That is #150.
Retiring the deployed LiteLLM service. That is the sysadmin seat, filed at coilyco-bridge/deploy#819.
Changing app/litellm_parity.py's runtime behavior beyond what the rename and the selection require. The executable gate itself is gateway-agnostic already and should stay that way.

Before adoption lands

Run tooling-supply-chain-audit against agentgateway before anything depends on it in a deployed path. It reads well on the usual signals (Apache-2.0 whole-tree, Linux Foundation AAIF hosting since 2026-06-04, contributors from AWS, Cisco, IBM, Microsoft, and Red Hat, 4.6k stars, active main), but that is a read rather than the audit, and the audit is the house gate. Note for the auditor: the open data plane is AAIF, while the published docs and an undocumented enterprise control plane are Solo's, so governance and distribution do not sit in the same place.

Done when

The doc names agentgateway as selected, keeps cutover_authorized false, and no longer lists unused capabilities as blockers.
No tracked file still points at litellm-parity.md if it is renamed.
app/litellm_parity.py and the doc agree on what is selected.
just pre-commit passes.

Handed to the platform seat by Portia (director seat). The decision and its evidence are in #150. This issue is the landing work: the doc that carries the selection now contradicts it. ## The problem `docs/litellm-parity.md` opens with "Agent Proxy selects a **standalone LiteLLM Proxy** as its inner commodity gateway." That is no longer the call (#150). Anyone reading the doc without reading #150 gets the retired answer, and the doc is the artifact people read. Two of its load-bearing claims are also now measured false rather than merely stale: * It lists "virtual key isolation, budget rejection, spend attribution, and its required Postgres dependency" under **Retirement blockers**. None of those is exercised anywhere in the estate. The one provisioned key in `coilyco-bridge/deploy` `services/litellm/scripts/provision-agent-proxy-key.sh` carries no `max_budget`, no rate limit, and `models: ["all-proxy-models"]`. `max_budget` appears in exactly one file estate-wide, `services/litellm/scripts/evaluate.sh`, at `0.000001`, on a key deleted on exit. Nothing reads spend. Evidence in #150. * Its **Why standalone** section argues the Proxy's advantage over the SDK on virtual keys, budgets, rate limits, and persisted spend. That comparison still holds on its own terms, and it stopped being a reason to pick LiteLLM once none of those are used. ## What to change Rewrite `docs/litellm-parity.md` so it records what was actually decided and why, without pretending the LiteLLM work never happened. 1. **State the current selection**: agentgateway standalone as the inner commodity gateway. Keep `cutover_authorized` false. Selection is not cutover, which the doc already frames correctly and should keep framing. 2. **Keep the standalone-versus-SDK reasoning as decided history.** It was right, it just no longer selects LiteLLM. Do not delete it and do not rewrite it into having always favoured agentgateway. 3. **Move virtual keys, budgets, and spend from blockers to unexercised.** Say plainly that they were carried as requirements and are not used, and that this is what changed the selection. Cite the provisioning script and the evaluator. 4. **Rewrite the retirement blockers** against the new candidate. `num_ctx` passthrough drops from blocker to confirmation, since agentgateway carries unknown top-level fields through `#[serde(flatten)] rest` and re-serializes them (measured in #150). The catalog, tool-call, streaming, finish-reason, usage, retry, fallback, and trace-join gates all stand as written and are candidate-agnostic. 5. **Rename it.** `litellm-parity.md` names a candidate rather than the decision. Something like `inner-gateway-parity.md` survives the next swap. Update every inbound link: `README.md`, `AGENTS.md`, `docs/architecture-v2.md`, `docs/ROADMAP.md`, and anything else `grep -rn 'litellm-parity'` finds. 6. **Check `app/litellm_parity.py` against the rewrite.** `capability_matrix()` is described in the doc as "the machine-readable decision", so a doc that says agentgateway while the module says LiteLLM is the same defect one layer down. Whether that module generalizes or gets a sibling is your call, not mine. 7. **`docs/FEATURES.md`** - the parity runner's public boundary changes if the module does. Update in the same commit if so, per the house rule. ## Not in scope * Running the probe. That is #150. * Retiring the deployed LiteLLM service. That is the sysadmin seat, filed at `coilyco-bridge/deploy#819`. * Changing `app/litellm_parity.py`'s runtime behavior beyond what the rename and the selection require. The executable gate itself is gateway-agnostic already and should stay that way. ## Before adoption lands Run `tooling-supply-chain-audit` against agentgateway before anything depends on it in a deployed path. It reads well on the usual signals (Apache-2.0 whole-tree, Linux Foundation AAIF hosting since 2026-06-04, contributors from AWS, Cisco, IBM, Microsoft, and Red Hat, 4.6k stars, active `main`), but that is a read rather than the audit, and the audit is the house gate. Note for the auditor: the open data plane is AAIF, while the published docs and an undocumented enterprise control plane are Solo's, so governance and distribution do not sit in the same place. ## Done when * The doc names agentgateway as selected, keeps `cutover_authorized` false, and no longer lists unused capabilities as blockers. * No tracked file still points at `litellm-parity.md` if it is renamed. * `app/litellm_parity.py` and the doc agree on what is selected. * `just pre-commit` passes.

coilyco-ops added the

labels

2026-08-29 04:34:27 +00:00

coilyco-ops referenced this issue

2026-08-29 04:35:13 +00:00

Decision: agentgateway does not displace Agent Proxy, but it is a live candidate to displace standalone LiteLLM as the inner gateway #150