Support truthful readiness for LiteLLM-only routes #79

New issue

Closed

opened 2026-08-05 00:59:30 +00:00 by coilyco-ops · 0 comments

coilyco-ops commented

2026-08-05 00:59:30 +00:00

Owner

Parent: coilysiren/inbox#329

Companion gate: coilyco-bridge/agentic-os-hardware#83

Outcome

Agent Proxy treats a governed LiteLLM route with direct: null as a valid hosted-provider route. Its readiness endpoint verifies the LiteLLM auth, health, and model catalog without pretending that an unrelated Ollama target proves the hosted provider is ready.

Scope

Preserve the current local-model readiness behavior for routes with Ollama readiness targets.
In LiteLLM mode, allow a route with no direct or readiness targets when the configured OpenAI-compatible upstream resolves through LiteLLM.
Check LiteLLM authentication, /health/readiness, and the logical route in /v1/models.
Skip Ollama version and catalog checks for a LiteLLM-only route.
Preserve fail-closed behavior in direct mode.
Add focused tests and update the readiness documentation.

Acceptance

A direct: null LiteLLM-only fixture reports ready when LiteLLM auth, health, and catalog checks pass.
The fixture performs no Ollama requests.
The same route reports not ready in direct mode.
Existing local-model readiness tests remain green.
No provider credentials, deployment mutation, or live inference belongs in this issue.

Parent: coilysiren/inbox#329 Companion gate: coilyco-bridge/agentic-os-hardware#83 ## Outcome Agent Proxy treats a governed LiteLLM route with `direct: null` as a valid hosted-provider route. Its readiness endpoint verifies the LiteLLM auth, health, and model catalog without pretending that an unrelated Ollama target proves the hosted provider is ready. ## Scope * Preserve the current local-model readiness behavior for routes with Ollama readiness targets. * In LiteLLM mode, allow a route with no direct or readiness targets when the configured OpenAI-compatible upstream resolves through LiteLLM. * Check LiteLLM authentication, `/health/readiness`, and the logical route in `/v1/models`. * Skip Ollama version and catalog checks for a LiteLLM-only route. * Preserve fail-closed behavior in direct mode. * Add focused tests and update the readiness documentation. ## Acceptance * A `direct: null` LiteLLM-only fixture reports ready when LiteLLM auth, health, and catalog checks pass. * The fixture performs no Ollama requests. * The same route reports not ready in direct mode. * Existing local-model readiness tests remain green. * No provider credentials, deployment mutation, or live inference belongs in this issue.

coilyco-ops added the

autonomy

headless

priority

labels

2026-08-05 00:59:36 +00:00

coilyco-ops referenced this issue from a commit

2026-08-05 01:05:11 +00:00

fix(readiness): support LiteLLM-only routes

coilyco-ops closed this issue

2026-08-05 01:05:11 +00:00