coilyco-flight-deck/agent-proxy

Watch

Consume a deploy-mounted logical route registry #62

New issue

Closed

opened 2026-07-28 01:45:07 +00:00 by coilyco-ops · 1 comment

coilyco-ops commented

2026-07-28 01:45:07 +00:00

Member

Outcome

Agent Proxy accepts model-opaque logical lane keys from governed clients and resolves them through a Deploy-owned mounted route registry. Concrete backend names remain behind the proxy boundary, while LiteLLM continues to own provider protocol translation, commodity routing, retries, and fallback.

Blocked by coilyco-bridge/agentic-os-hardware#79

Current evidence

The live Agent Proxy and LiteLLM workloads are already healthy. Agent Proxy currently advertises physical backend names such as ornith:35b and qwen3-coder:30b. That proves the transport path, but it couples clients to deployment choices and bypasses the role-and-intent ownership held by AOSH.

Ownership boundary

Agent Proxy owns policy enforcement, context handling, route-registry validation, logical-to-upstream dispatch, and route-safe telemetry.
LiteLLM owns provider protocols, retries, provider fallback, and physical backend selection from its deployed configuration.
Deploy owns the mounted service configuration and turns the AOSH source contract into Agent Proxy's local registry shape.
Agent Proxy does not clone, import, or fetch AOSH.
Agent Proxy does not choose which model belongs to a role or intent.
A role or logical route is control-plane data. Agent Proxy must not inject it into messages, prompts, or provider payload fields that become model context.

Registry contract

Implement a small, versioned local schema with this semantic shape:

format: agent-proxy-route-registry/v1
One record per stable <role>/<intent> key.
Each record names the LiteLLM model alias Agent Proxy sends upstream.
Each record may carry a direct-mode target that Agent Proxy can use during the established LiteLLM rollback path.
Direct-mode target data is deployment data, not an AOSH import. It is generated and mounted by Deploy.

The implementation may choose YAML or JSON according to existing repository conventions. The schema and loader must be independent of AOSH's repository layout and source filename.

Request behavior

Authenticated /v1/models returns logical lane keys, not physical backend names.
An OpenAI-compatible request uses the logical lane key in the model field.
In LiteLLM mode, Agent Proxy validates the key and forwards the configured LiteLLM alias. The normal case should preserve the same logical key end to end.
In direct rollback mode, Agent Proxy resolves a supported direct target from the mounted registry.
Direct mode fails closed with a clear non-secret error when a lane requires a runtime Agent Proxy does not support directly.
Unknown, duplicate, disabled, or malformed routes fail deterministically.
Startup should fail before serving traffic when the configured registry is unreadable or invalid. An explicit empty-registry compatibility mode is acceptable only if current startup behavior requires it and the mode is documented.

Telemetry and safety

Traces and metrics record the logical route key and the selected upstream mode.
Physical backend identity may appear only in existing restricted operational telemetry where it is required to diagnose the upstream.
Logs and errors do not expose credentials, endpoints, or raw customer payloads.
Route metadata is never appended to chat content or system prompts.

Acceptance criteria

Loader tests cover a valid registry, an unknown format, duplicate keys, missing upstream aliases, unsafe file handling, and malformed direct targets.
API tests prove /v1/models exposes logical keys and hides concrete backend names.
Request tests prove a logical key reaches the configured LiteLLM alias.
Direct-mode tests prove supported targets resolve and unsupported runtime targets fail closed without silently choosing a different model.
Context-safety tests prove route and role metadata never enters provider message content.
Existing authentication, streaming, trajectory persistence, and LiteLLM rollback behavior remain covered.
Documentation identifies the registry mount, startup behavior, rollback behavior, and ownership boundary.
Repository validation passes and the change lands on canonical main.

Non-goals

Do not embed AOSH files or parsers in Agent Proxy.
Do not add a runtime dependency on AOSH or Agent Compose.
Do not reimplement LiteLLM's retry and fallback engine.
Do not expose physical models as the governed client contract.
Do not perform a live rollout in this issue.

## Outcome Agent Proxy accepts model-opaque logical lane keys from governed clients and resolves them through a Deploy-owned mounted route registry. Concrete backend names remain behind the proxy boundary, while LiteLLM continues to own provider protocol translation, commodity routing, retries, and fallback. Blocked by https://forgejo.coilysiren.me/coilyco-bridge/agentic-os-hardware/issues/79 ## Current evidence The live Agent Proxy and LiteLLM workloads are already healthy. Agent Proxy currently advertises physical backend names such as `ornith:35b` and `qwen3-coder:30b`. That proves the transport path, but it couples clients to deployment choices and bypasses the role-and-intent ownership held by AOSH. ## Ownership boundary * Agent Proxy owns policy enforcement, context handling, route-registry validation, logical-to-upstream dispatch, and route-safe telemetry. * LiteLLM owns provider protocols, retries, provider fallback, and physical backend selection from its deployed configuration. * Deploy owns the mounted service configuration and turns the AOSH source contract into Agent Proxy's local registry shape. * Agent Proxy does not clone, import, or fetch AOSH. * Agent Proxy does not choose which model belongs to a role or intent. * A role or logical route is control-plane data. Agent Proxy must not inject it into messages, prompts, or provider payload fields that become model context. ## Registry contract Implement a small, versioned local schema with this semantic shape: * `format: agent-proxy-route-registry/v1` * One record per stable `<role>/<intent>` key. * Each record names the LiteLLM model alias Agent Proxy sends upstream. * Each record may carry a direct-mode target that Agent Proxy can use during the established LiteLLM rollback path. * Direct-mode target data is deployment data, not an AOSH import. It is generated and mounted by Deploy. The implementation may choose YAML or JSON according to existing repository conventions. The schema and loader must be independent of AOSH's repository layout and source filename. ## Request behavior * Authenticated `/v1/models` returns logical lane keys, not physical backend names. * An OpenAI-compatible request uses the logical lane key in the `model` field. * In LiteLLM mode, Agent Proxy validates the key and forwards the configured LiteLLM alias. The normal case should preserve the same logical key end to end. * In direct rollback mode, Agent Proxy resolves a supported direct target from the mounted registry. * Direct mode fails closed with a clear non-secret error when a lane requires a runtime Agent Proxy does not support directly. * Unknown, duplicate, disabled, or malformed routes fail deterministically. * Startup should fail before serving traffic when the configured registry is unreadable or invalid. An explicit empty-registry compatibility mode is acceptable only if current startup behavior requires it and the mode is documented. ## Telemetry and safety * Traces and metrics record the logical route key and the selected upstream mode. * Physical backend identity may appear only in existing restricted operational telemetry where it is required to diagnose the upstream. * Logs and errors do not expose credentials, endpoints, or raw customer payloads. * Route metadata is never appended to chat content or system prompts. ## Acceptance criteria * Loader tests cover a valid registry, an unknown format, duplicate keys, missing upstream aliases, unsafe file handling, and malformed direct targets. * API tests prove `/v1/models` exposes logical keys and hides concrete backend names. * Request tests prove a logical key reaches the configured LiteLLM alias. * Direct-mode tests prove supported targets resolve and unsupported runtime targets fail closed without silently choosing a different model. * Context-safety tests prove route and role metadata never enters provider message content. * Existing authentication, streaming, trajectory persistence, and LiteLLM rollback behavior remain covered. * Documentation identifies the registry mount, startup behavior, rollback behavior, and ownership boundary. * Repository validation passes and the change lands on canonical `main`. ## Non-goals * Do not embed AOSH files or parsers in Agent Proxy. * Do not add a runtime dependency on AOSH or Agent Compose. * Do not reimplement LiteLLM's retry and fallback engine. * Do not expose physical models as the governed client contract. * Do not perform a live rollout in this issue.

coilyco-ops added the

autonomy

headless

priority

labels

2026-07-28 01:45:12 +00:00

coilyco-ops referenced this issue from coilyco-flight-deck/agentic-os

2026-07-28 01:46:06 +00:00

Emit model-opaque logical lanes for harness adapters #767

coilysiren referenced this issue from a commit

2026-07-28 02:25:37 +00:00

feat: consume logical route registry

coilysiren closed this issue

2026-07-28 02:25:37 +00:00

coilyco-ops commented

2026-07-28 02:25:49 +00:00

Author

Member

Landed on canonical main in 87762a79aa944a445d0a4399ff93cae4bf4a33b4.

Evidence:

ward exec test passes 190 tests, including strict registry loader, logical catalog, LiteLLM alias forwarding, supported direct target, unsupported runtime, disabled route, and context-isolation coverage.
ward exec format-check, ward exec lint, and ward exec typecheck pass.
ward exec boot-probe and ward exec smoke pass against the frozen runtime dependency set.
A configured invalid registry fails during lifespan startup before the queue or traffic-serving path starts.
/v1/models and MCP discovery expose enabled logical keys only in registry mode.
Traces and llm_route_requests_total carry the logical key and selected upstream mode. Physical targets remain inside backend dispatch.
The documented compatibility flag preserves the legacy no-registry development surface. Deploy can disable it when mounting the production registry.

Landed on canonical `main` in `87762a79aa944a445d0a4399ff93cae4bf4a33b4`. Evidence: * `ward exec test` passes 190 tests, including strict registry loader, logical catalog, LiteLLM alias forwarding, supported direct target, unsupported runtime, disabled route, and context-isolation coverage. * `ward exec format-check`, `ward exec lint`, and `ward exec typecheck` pass. * `ward exec boot-probe` and `ward exec smoke` pass against the frozen runtime dependency set. * A configured invalid registry fails during lifespan startup before the queue or traffic-serving path starts. * `/v1/models` and MCP discovery expose enabled logical keys only in registry mode. * Traces and `llm_route_requests_total` carry the logical key and selected upstream mode. Physical targets remain inside backend dispatch. * The documented compatibility flag preserves the legacy no-registry development surface. Deploy can disable it when mounting the production registry.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

coilyco-flight-deck/agent-proxy#62

No description provided.

Rows
Columns