coilyco-flight-deck/agent-proxy

Watch

Decision: agentgateway does not displace Agent Proxy, but it is a live candidate to displace standalone LiteLLM as the inner gateway #150

New issue

Open

opened 2026-08-29 04:16:54 +00:00 by coilyco-ops · 3 comments

coilyco-ops commented

2026-08-29 04:16:54 +00:00

Owner

Call 1 - Agent Proxy is not deprecated. Closed.

The comparison is category-mismatched. agentgateway is a commodity gateway. Agent Proxy already decided the commodity-gateway half is not its value and delegates it downward.

README.md on main (36d9e73): "LiteLLM will own provider protocols, routing, retry, fallback, keys, budgets, and cost accounting." What Agent Proxy keeps is the evidence plane - and the README is equally explicit that "SigNoz and OTLP carry operational traces, logs, and metrics, and are deliberately not the durable training-data store."

agentgateway emits OpenTelemetry metrics, logs, and tracing. That is the SigNoz-shaped artifact the architecture already declines as the durable store. It does not ship the trajectory contract, append-only ingestion with idempotent receipts and replay, deterministic episode reconstruction with provenance and content hashes, immutable evaluation records with supersession, or dataset export with write-once manifests and trajectory-level leakage prevention.

Size of what a deprecation would discard, counted on main: app/trajectory/ is 4,072 lines, plus obs.py 622, skill_use.py 455, analysis.py 303, body_capture.py 236. Roughly 5.7K of ~10.9K app lines have no counterpart in agentgateway.

Reverses if something ships durable trajectory-contract-shaped capture - episode reconstruction with provenance hashes, immutable evaluation records, leakage-preventing dataset export. OTel telemetry is not that artifact and adding more of it does not become that artifact.

Call 2 - agentgateway vs standalone LiteLLM as the inner gateway. Open, and worth one bounded probe.

docs/litellm-parity.md selects standalone LiteLLM but holds cutover_authorized false pending the retirement blockers. Nothing is deployed on that decision yet, so the switching cost today is the client shim (app/litellm_parity.py, 418 lines) rather than a migration.

The parity gate is already built and gateway-agnostic: just litellm-parity --baseline-url URL --candidate-url URL --model TAG. Pointing it at an agentgateway standalone binary is cheap.

For agentgateway:

One binary, no database on the request path. litellm-parity.md lists "virtual key isolation, budget rejection, spend attribution, and its required Postgres dependency" as a retirement blocker. agentgateway deletes that blocker rather than satisfying it.
Apache-2.0 across the whole tree. LiteLLM gates SSO, OIDC, and vault integration behind its enterprise tier.
Ollama is a first-class provider in agentgateway 1.3 standalone. The estate runs Ollama towers.
Native MCP federation, next to the /mcp surface Agent Proxy already ships.

Against, or unweighed:

The headline benchmark (37k QPS vs 3.2k, 12GB vs tens of MB) is agentgateway's own June 2026 blog against a mock upstream. It buys nothing here - #128 puts sirens-echo/default at 276 calls a week, and the bottleneck is an Ollama tower generating tokens, not gateway QPS.
The one recommendation source (fmind.dev/articles/agentgateway-vs-litellm/, 2026-08-16) discloses AAIF Ambassador status. Its 2026 advisory count - agentgateway 3 with 1 High, LiteLLM 12 with 3 Critical and 2 in CISA KEV - is single-sourced and not independently verified.
The published docs largely live on docs.solo.io as "Solo Enterprise for agentgateway", with an undocumented enterprise control plane. Open data plane, vendor-shaped docs and control plane.

The deciding blocker, genuinely open: whether agentgateway passes Agent Proxy's derived safe num_ctx through to Ollama as a top-level extension. Agent Proxy forwards it and then verifies delivered context; the LiteLLM boundary already accommodates that. agentgateway's provider docs cover base URL and auth passthrough and say nothing about unknown-parameter passthrough. If num_ctx does not survive the hop, agentgateway is disqualified as the inner gateway regardless of everything above.

The probe

Owned by the platform seat. Bounded, and it stops at evidence rather than at a cutover.

Stand up agentgateway standalone against a real tower with the ollama provider.
Send a request carrying a top-level num_ctx and assert delivered-context verification stays green. Stop here if it fails and record the disqualification.
Run just litellm-parity with agentgateway as candidate and the current surface as baseline. Keep the JSON artifact.
Report virtual-key, budget, and spend-attribution coverage against the litellm-parity.md standalone-advantage list.

Existing blockers stay where they are. cutover_authorized stays false either way - this probe changes which candidate the gate is aimed at, not whether the gate is passed.

Related: #134 and #129 (cancellation not propagating upstream), #111 (automatic fallback model), #107 (queue or shed under burst). Those are commodity-gateway defects in the hot path, and they are the standing argument for finishing this hand-off to some inner gateway rather than growing the custom one.

Filed by Portia (director seat) from a session where Kai raised deprecating Agent Proxy in favour of agentgateway. Recording the call so nobody re-derives it, and scoping the one probe that is actually live. Sub-answer to `coilysiren/inbox#428` (Scout everything AAIF). ## Call 1 - Agent Proxy is not deprecated. Closed. The comparison is category-mismatched. agentgateway is a commodity gateway. Agent Proxy already decided the commodity-gateway half is not its value and delegates it downward. `README.md` on `main` (36d9e73): "LiteLLM will own provider protocols, routing, retry, fallback, keys, budgets, and cost accounting." What Agent Proxy keeps is the evidence plane - and the README is equally explicit that "SigNoz and OTLP carry operational traces, logs, and metrics, and are deliberately not the durable training-data store." agentgateway emits OpenTelemetry metrics, logs, and tracing. That is the SigNoz-shaped artifact the architecture already declines as the durable store. It does not ship the trajectory contract, append-only ingestion with idempotent receipts and replay, deterministic episode reconstruction with provenance and content hashes, immutable evaluation records with supersession, or dataset export with write-once manifests and trajectory-level leakage prevention. Size of what a deprecation would discard, counted on `main`: `app/trajectory/` is 4,072 lines, plus `obs.py` 622, `skill_use.py` 455, `analysis.py` 303, `body_capture.py` 236. Roughly 5.7K of ~10.9K app lines have no counterpart in agentgateway. **Reverses if** something ships durable trajectory-contract-shaped capture - episode reconstruction with provenance hashes, immutable evaluation records, leakage-preventing dataset export. OTel telemetry is not that artifact and adding more of it does not become that artifact. ## Call 2 - agentgateway vs standalone LiteLLM as the inner gateway. Open, and worth one bounded probe. `docs/litellm-parity.md` selects standalone LiteLLM but holds `cutover_authorized` false pending the retirement blockers. Nothing is deployed on that decision yet, so the switching cost today is the client shim (`app/litellm_parity.py`, 418 lines) rather than a migration. The parity gate is already built and gateway-agnostic: `just litellm-parity --baseline-url URL --candidate-url URL --model TAG`. Pointing it at an agentgateway standalone binary is cheap. **For agentgateway:** * One binary, no database on the request path. `litellm-parity.md` lists "virtual key isolation, budget rejection, spend attribution, and its required Postgres dependency" as a retirement blocker. agentgateway deletes that blocker rather than satisfying it. * Apache-2.0 across the whole tree. LiteLLM gates SSO, OIDC, and vault integration behind its enterprise tier. * Ollama is a first-class provider in agentgateway 1.3 standalone. The estate runs Ollama towers. * Native MCP federation, next to the `/mcp` surface Agent Proxy already ships. **Against, or unweighed:** * The headline benchmark (37k QPS vs 3.2k, 12GB vs tens of MB) is agentgateway's own June 2026 blog against a mock upstream. It buys nothing here - `#128` puts `sirens-echo/default` at 276 calls a week, and the bottleneck is an Ollama tower generating tokens, not gateway QPS. * The one recommendation source (`fmind.dev/articles/agentgateway-vs-litellm/`, 2026-08-16) discloses AAIF Ambassador status. Its 2026 advisory count - agentgateway 3 with 1 High, LiteLLM 12 with 3 Critical and 2 in CISA KEV - is single-sourced and not independently verified. * The published docs largely live on `docs.solo.io` as "Solo Enterprise for agentgateway", with an undocumented enterprise control plane. Open data plane, vendor-shaped docs and control plane. **The deciding blocker, genuinely open:** whether agentgateway passes Agent Proxy's derived safe `num_ctx` through to Ollama as a top-level extension. Agent Proxy forwards it and then verifies delivered context; the LiteLLM boundary already accommodates that. agentgateway's provider docs cover base URL and auth passthrough and say nothing about unknown-parameter passthrough. If `num_ctx` does not survive the hop, agentgateway is disqualified as the inner gateway regardless of everything above. ## The probe Owned by the platform seat. Bounded, and it stops at evidence rather than at a cutover. 1. Stand up agentgateway standalone against a real tower with the `ollama` provider. 2. Send a request carrying a top-level `num_ctx` and assert delivered-context verification stays green. **Stop here if it fails** and record the disqualification. 3. Run `just litellm-parity` with agentgateway as candidate and the current surface as baseline. Keep the JSON artifact. 4. Report virtual-key, budget, and spend-attribution coverage against the `litellm-parity.md` standalone-advantage list. Existing blockers stay where they are. `cutover_authorized` stays false either way - this probe changes which candidate the gate is aimed at, not whether the gate is passed. Related: `#134` and `#129` (cancellation not propagating upstream), `#111` (automatic fallback model), `#107` (queue or shed under burst). Those are commodity-gateway defects in the hot path, and they are the standing argument for finishing this hand-off to *some* inner gateway rather than growing the custom one.

coilyco-ops commented

2026-08-29 04:27:14 +00:00

Author

Owner

Correction: the num_ctx blocker is dead. agentgateway passes unknown top-level parameters through. I read the source instead of the docs, then measured the round-trip.

Checked at agentgateway/agentgateway@5fb188b.

The mechanism

crates/llm/src/types/completions.rs, the OpenAI chat-completions request type, ends with a catch-all:

pub struct Request {
	pub messages: Vec<RequestMessage>,
	...
	#[serde(flatten, default)]
	pub rest: serde_json::Value,
}

Unknown top-level fields collect into rest on deserialize and are re-emitted on serialize. RequestMessage carries the same rest field, so unknown per-message fields survive too. crates/llm/src/conversion/vertex.rs:87 names the intent outright: "output_dimensionality has no OpenAI equivalent and arrives via the passthrough rest."

The outbound render for our path does no filtering, crates/agentgateway/src/llm/mod.rs:355:

types::ChatRequest::Completions(mut req) => {
	apply_openai_moderation(&mut req.moderation, ctx)?;
	serde_json::to_vec(&req).map_err(AIError::RequestMarshal)
},

Ollama is ProviderPreset::Ollama in crates/llm/src/custom.rs, so it routes as AIProvider::Custom on the OpenAI completions format and lands on exactly that render.

The measurement

Compiled a probe mirroring that struct field-for-field on the attributes that decide the outcome, and ran an Agent-Proxy-shaped body through deserialize then serialize:

rest captured = {"num_ctx":16384,"options":{"num_ctx":16384},"x_ward_correlation":"abc-123"}
outbound      = {"messages":[...],"model":"ornith:35b","temperature":0.2,"num_ctx":16384,"options":{"num_ctx":16384},"x_ward_correlation":"abc-123"}
  num_ctx              survives = true
  options              survives = true
  x_ward_correlation   survives = true
negative control (no flatten): num_ctx survives = false

Top-level num_ctx, nested options.num_ctx, and an arbitrary correlation key all survive. The negative control, the same struct with the flatten field removed, drops all three, which is the failure mode I had assumed and is not what agentgateway does.

What this does and does not settle

Settled: agentgateway does not silently strip Agent Proxy's derived safe num_ctx, and it does not strip arbitrary correlation fields either. No disqualification.

Not settled, and the probe still runs to confirm:

I mirrored the struct rather than compiling agentgateway's own crate. Attributes copied field-for-field on the deciding ones, but that is a model of the code, not the code.
apply_model_alias mutates model under policy on this path. It does not touch rest, but policy-driven body mutation exists and delivered-context verification should still be asserted end to end.
Nothing here measures whether Ollama then honours num_ctx, only that it arrives.

Probe revised

Step 2 stops being a stop-or-go gate and becomes confirmation. Reordered:

Stand up agentgateway standalone against a real tower with the ollama provider.
Run just litellm-parity with agentgateway as candidate against the current surface as baseline. Keep the JSON artifact.
Confirm num_ctx arrives and delivered-context verification stays green through the live hop.
Report virtual-key, budget, and spend-attribution coverage against the litellm-parity.md standalone-advantage list. This is now the largest open question, since it is where LiteLLM's standalone advantage was argued.

Effect on the calls

Call 1 is unchanged. Agent Proxy is still not deprecated, and none of this touches the trajectory plane.

Call 2 moves from open to leaning. The disqualifying risk is gone and the remaining case favours agentgateway on operational surface: one binary against FastAPI plus Postgres plus Redis, where litellm-parity.md lists that Postgres dependency as a retirement blocker in its own right. That is the reason to prefer it. The QPS benchmark is still irrelevant at 276 calls a week and the AAIF hosting is still not evidence.

**Correction: the `num_ctx` blocker is dead. agentgateway passes unknown top-level parameters through.** I read the source instead of the docs, then measured the round-trip. Checked at `agentgateway/agentgateway@5fb188b`. ## The mechanism `crates/llm/src/types/completions.rs`, the OpenAI chat-completions request type, ends with a catch-all: ```rust pub struct Request { pub messages: Vec<RequestMessage>, ... #[serde(flatten, default)] pub rest: serde_json::Value, } ``` Unknown top-level fields collect into `rest` on deserialize and are re-emitted on serialize. `RequestMessage` carries the same `rest` field, so unknown per-message fields survive too. `crates/llm/src/conversion/vertex.rs:87` names the intent outright: "`output_dimensionality` has no OpenAI equivalent and arrives via the passthrough `rest`." The outbound render for our path does no filtering, `crates/agentgateway/src/llm/mod.rs:355`: ```rust types::ChatRequest::Completions(mut req) => { apply_openai_moderation(&mut req.moderation, ctx)?; serde_json::to_vec(&req).map_err(AIError::RequestMarshal) }, ``` Ollama is `ProviderPreset::Ollama` in `crates/llm/src/custom.rs`, so it routes as `AIProvider::Custom` on the OpenAI completions format and lands on exactly that render. ## The measurement Compiled a probe mirroring that struct field-for-field on the attributes that decide the outcome, and ran an Agent-Proxy-shaped body through deserialize then serialize: ``` rest captured = {"num_ctx":16384,"options":{"num_ctx":16384},"x_ward_correlation":"abc-123"} outbound = {"messages":[...],"model":"ornith:35b","temperature":0.2,"num_ctx":16384,"options":{"num_ctx":16384},"x_ward_correlation":"abc-123"} num_ctx survives = true options survives = true x_ward_correlation survives = true negative control (no flatten): num_ctx survives = false ``` Top-level `num_ctx`, nested `options.num_ctx`, and an arbitrary correlation key all survive. The negative control, the same struct with the flatten field removed, drops all three, which is the failure mode I had assumed and is not what agentgateway does. ## What this does and does not settle Settled: agentgateway does not silently strip Agent Proxy's derived safe `num_ctx`, and it does not strip arbitrary correlation fields either. No disqualification. Not settled, and the probe still runs to confirm: * I mirrored the struct rather than compiling agentgateway's own crate. Attributes copied field-for-field on the deciding ones, but that is a model of the code, not the code. * `apply_model_alias` mutates `model` under policy on this path. It does not touch `rest`, but policy-driven body mutation exists and delivered-context verification should still be asserted end to end. * Nothing here measures whether Ollama then honours `num_ctx`, only that it arrives. ## Probe revised Step 2 stops being a stop-or-go gate and becomes confirmation. Reordered: 1. Stand up agentgateway standalone against a real tower with the `ollama` provider. 2. Run `just litellm-parity` with agentgateway as candidate against the current surface as baseline. Keep the JSON artifact. 3. Confirm `num_ctx` arrives and delivered-context verification stays green through the live hop. 4. Report virtual-key, budget, and spend-attribution coverage against the `litellm-parity.md` standalone-advantage list. This is now the largest open question, since it is where LiteLLM's standalone advantage was argued. ## Effect on the calls Call 1 is unchanged. Agent Proxy is still not deprecated, and none of this touches the trajectory plane. Call 2 moves from open to leaning. The disqualifying risk is gone and the remaining case favours agentgateway on operational surface: one binary against FastAPI plus Postgres plus Redis, where `litellm-parity.md` lists that Postgres dependency as a retirement blocker in its own right. That is the reason to prefer it. The QPS benchmark is still irrelevant at 276 calls a week and the AAIF hosting is still not evidence.

coilyco-ops commented

2026-08-29 04:31:29 +00:00

Author

Owner

Second correction, and it closes Call 2. The estate does not use virtual keys or budgets. LiteLLM's entire standalone advantage is unexercised.

I carried that from litellm-parity.md's capability list and called it the largest open question. It was a feature list, not usage. Kai asked whether we actually use it. We do not. Checked coilyco-bridge/deploy@main.

The production key has no budget and no scope

services/litellm/scripts/provision-agent-proxy-key.sh provisions the one durable key at /coilysiren/litellm/agent-proxy-key:

{
  "key_alias": "agent-proxy-ser8",
  "models": ["all-proxy-models"],
  "metadata": { "purpose": "agent-proxy-ser8-inner-gateway" }
}

No max_budget. No tpm_limit or rpm_limit. models: ["all-proxy-models"] is access to everything. That is a bearer token with an alias attached, and any gateway that checks an Authorization header does the same job.

`max_budget` exists in exactly one file, and it is a test fixture

Across the whole service, max_budget, budget_duration, soft_budget, tpm_limit, and rpm_limit appear only in services/litellm/scripts/evaluate.sh, at 0.000001, on a key the evaluator creates to prove rejection fires and then deletes on exit. just evaluate-litellm-ser8 proves the feature works. Nothing runs it in production.

Nothing reads the spend

Grepped both repos for /spend, spend/logs, global/spend, and LiteLLM_SpendLogs. Zero hits outside the evaluator. The only /key/info call is the provisioner checking its own key is still valid.

And the numbers would be fiction anyway for most traffic. chart/templates/configmap.yaml applies synthetic_accounting with flat inputCostPerToken and outputCostPerToken to the local providers, so spend on the Ollama fleet is a made-up rate. Only hosted DeepSeek carries real provider accounting, and nothing queries it.

Cost facts that Agent Proxy actually consumes arrive in the response and land in the trajectory-contract-v1 model-execution fields. They do not come from LiteLLM's Postgres.

So Postgres is carrying nothing

services/litellm/README.md puts PostgreSQL 16.14 on a 5 Gi local-path claim to retain "spend and key state". The key state is one row. The spend state has no reader. That is a stateful dependency, a PVC, and a backup surface on ser8, held up entirely by a capability nobody exercises.

litellm-parity.md lists "virtual key isolation, budget rejection, spend attribution, and its required Postgres dependency" as a retirement blocker. It was never a blocker. It was a cost.

Call 2 is closed: agentgateway is selected as the inner gateway

Deciding evidence, all of it now in:

num_ctx and arbitrary correlation fields pass through agentgateway. Measured, previous comment.
Virtual keys, budgets, and spend accounting, the only things LiteLLM's standalone mode won on, are unused here. Measured above.
The current route set is covered. Deepseek and Ollama are both ProviderPreset values in crates/llm/src/custom.rs, so hosted DeepSeek and the local towers both route natively.
What is left is one Rust binary against FastAPI plus PostgreSQL plus a PVC, serving one consumer at roughly 276 calls a week.

Foreclosed: standalone LiteLLM as the inner gateway, and with it the Postgres and PVC on ser8. services/litellm/ becomes a retirement candidate rather than an evaluation candidate, but not before the parity gate passes on the replacement. Deleting it is the sysadmin seat's call and a separate issue.

Reverses if the parity gate fails on agentgateway, or a second independent consumer of the inner gateway appears and needs real per-tenant budget enforcement. One consumer is why none of this was ever needed.

Unchanged: cutover_authorized stays false. Selection is not cutover, exactly as the parity doc already frames it. The probe still runs, now as the gate on a chosen candidate rather than a bake-off.

Also unchanged: Call 1. Agent Proxy is not deprecated, and none of this touches the trajectory plane.

**Second correction, and it closes Call 2. The estate does not use virtual keys or budgets. LiteLLM's entire standalone advantage is unexercised.** I carried that from `litellm-parity.md`'s capability list and called it the largest open question. It was a feature list, not usage. Kai asked whether we actually use it. We do not. Checked `coilyco-bridge/deploy@main`. ## The production key has no budget and no scope `services/litellm/scripts/provision-agent-proxy-key.sh` provisions the one durable key at `/coilysiren/litellm/agent-proxy-key`: ```json { "key_alias": "agent-proxy-ser8", "models": ["all-proxy-models"], "metadata": { "purpose": "agent-proxy-ser8-inner-gateway" } } ``` No `max_budget`. No `tpm_limit` or `rpm_limit`. `models: ["all-proxy-models"]` is access to everything. That is a bearer token with an alias attached, and any gateway that checks an `Authorization` header does the same job. ## `max_budget` exists in exactly one file, and it is a test fixture Across the whole service, `max_budget`, `budget_duration`, `soft_budget`, `tpm_limit`, and `rpm_limit` appear only in `services/litellm/scripts/evaluate.sh`, at `0.000001`, on a key the evaluator creates to prove rejection fires and then deletes on exit. `just evaluate-litellm-ser8` proves the feature works. Nothing runs it in production. ## Nothing reads the spend Grepped both repos for `/spend`, `spend/logs`, `global/spend`, and `LiteLLM_SpendLogs`. Zero hits outside the evaluator. The only `/key/info` call is the provisioner checking its own key is still valid. And the numbers would be fiction anyway for most traffic. `chart/templates/configmap.yaml` applies `synthetic_accounting` with flat `inputCostPerToken` and `outputCostPerToken` to the local providers, so spend on the Ollama fleet is a made-up rate. Only hosted DeepSeek carries real provider accounting, and nothing queries it. Cost facts that Agent Proxy actually consumes arrive in the response and land in the `trajectory-contract-v1` model-execution fields. They do not come from LiteLLM's Postgres. ## So Postgres is carrying nothing `services/litellm/README.md` puts PostgreSQL 16.14 on a 5 Gi `local-path` claim to retain "spend and key state". The key state is one row. The spend state has no reader. That is a stateful dependency, a PVC, and a backup surface on ser8, held up entirely by a capability nobody exercises. `litellm-parity.md` lists "virtual key isolation, budget rejection, spend attribution, and its required Postgres dependency" as a retirement blocker. It was never a blocker. It was a cost. ## Call 2 is closed: agentgateway is selected as the inner gateway Deciding evidence, all of it now in: * `num_ctx` and arbitrary correlation fields pass through agentgateway. Measured, previous comment. * Virtual keys, budgets, and spend accounting, the only things LiteLLM's standalone mode won on, are unused here. Measured above. * The current route set is covered. `Deepseek` and `Ollama` are both `ProviderPreset` values in `crates/llm/src/custom.rs`, so hosted DeepSeek and the local towers both route natively. * What is left is one Rust binary against FastAPI plus PostgreSQL plus a PVC, serving one consumer at roughly 276 calls a week. **Foreclosed:** standalone LiteLLM as the inner gateway, and with it the Postgres and PVC on ser8. `services/litellm/` becomes a retirement candidate rather than an evaluation candidate, but not before the parity gate passes on the replacement. Deleting it is the sysadmin seat's call and a separate issue. **Reverses if** the parity gate fails on agentgateway, or a second independent consumer of the inner gateway appears and needs real per-tenant budget enforcement. One consumer is why none of this was ever needed. **Unchanged:** `cutover_authorized` stays false. Selection is not cutover, exactly as the parity doc already frames it. The probe still runs, now as the gate on a chosen candidate rather than a bake-off. Also unchanged: Call 1. Agent Proxy is not deprecated, and none of this touches the trajectory plane.

coilyco-ops referenced this issue

2026-08-29 04:34:27 +00:00

docs/litellm-parity.md still records standalone LiteLLM as selected, and the decision has moved to agentgateway #151

coilyco-ops commented

2026-08-29 04:35:13 +00:00

Author

Owner

Decision handed off. This issue stays the decision record and holds the probe.

#151 - rewrite docs/litellm-parity.md to record the agentgateway selection, rename it off the candidate's name, and reconcile app/litellm_parity.py. Platform seat, P2, headless. The doc currently states the retired answer, which is why it is P2 rather than P3.
coilyco-bridge/deploy#819 - retire services/litellm/ on ser8 and drop the Postgres and its 5 Gi PVC. Sysadmin seat, P3, blocked on the probe here and on an agentgateway service actually serving. Staged disable-observe-delete, with the PVC deletion held back as its own confirmation.

One gap, named rather than filed. #819 is blocked on "an agentgateway service exists in this repo and is serving Agent Proxy", and no issue owns building that yet. It is deliberately sequenced after the probe, because the deployment shape (config format, listener, secret paths, SigNoz pipeline) is not knowable until the probe reports. Whoever closes the probe files that issue as its immediate successor. It is the one link in this chain with no owner, so it is the one most likely to be dropped.

Decision handed off. This issue stays the decision record and holds the probe. * **#151** - rewrite `docs/litellm-parity.md` to record the agentgateway selection, rename it off the candidate's name, and reconcile `app/litellm_parity.py`. Platform seat, P2, headless. The doc currently states the retired answer, which is why it is P2 rather than P3. * **`coilyco-bridge/deploy#819`** - retire `services/litellm/` on ser8 and drop the Postgres and its 5 Gi PVC. Sysadmin seat, P3, blocked on the probe here and on an agentgateway service actually serving. Staged disable-observe-delete, with the PVC deletion held back as its own confirmation. **One gap, named rather than filed.** #819 is blocked on "an agentgateway service exists in this repo and is serving Agent Proxy", and no issue owns building that yet. It is deliberately sequenced after the probe, because the deployment shape (config format, listener, secret paths, SigNoz pipeline) is not knowable until the probe reports. Whoever closes the probe files that issue as its immediate successor. It is the one link in this chain with no owner, so it is the one most likely to be dropped.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

coilyco-flight-deck/agent-proxy#150

No description provided.

Rows
Columns

Decision: agentgateway does not displace Agent Proxy, but it is a live candidate to displace standalone LiteLLM as the inner gateway #150

Call 1 - Agent Proxy is not deprecated. Closed.

Call 2 - agentgateway vs standalone LiteLLM as the inner gateway. Open, and worth one bounded probe.

The probe

The mechanism

The measurement

What this does and does not settle

Probe revised

Effect on the calls

The production key has no budget and no scope

max_budget exists in exactly one file, and it is a test fixture

Nothing reads the spend

So Postgres is carrying nothing

Call 2 is closed: agentgateway is selected as the inner gateway

`max_budget` exists in exactly one file, and it is a test fixture