coilyco-flight-deck/agent-proxy
Watch
2
0
Code Issues 11 Pull requests Projects Releases Packages 1 Wiki Activity Actions

Reset Agent Proxy as a LiteLLM-backed trajectory and observability platform #40

New issue
Closed
opened 2026-07-23 05:30:12 +00:00 by coilyco-ops · 3 comments
coilyco-ops commented 2026-07-23 05:30:12 +00:00
Member
Copy link

Goal

Reset the Agent Proxy repository charter around the direction captured in coilyco-bridge/agentic-os-hardware#36:

Agent Proxy is the observation, trajectory collection, and data-processing plane for the agentic operations stack. LiteLLM supplies the commodity inference gateway beneath it.

This is a new work item. Historical issues remain closed evidence and must not be reopened or repurposed.

Why now

The repository has delivered a useful reliability proxy, but its current identity and roadmap lag the later product thesis:

  • There is no LiteLLM dependency.
  • models.py, upstream.py, queue.py, and resilience.py implement substantial commodity gateway behavior.
  • The service remains stateless apart from its in-memory queue.
  • Ward correlation metadata and skill-use artifact ingestion have landed, but the latter reduces records to logs and a Prometheus counter rather than persisting trajectories.
  • docs/ROADMAP.md still places data persistence in an unspecified future phase.
  • coilysiren/inbox#233 states the intended role directly: Agent Proxy is the dataset builder.

Scope

This issue is the repository-level architecture and work-graph reset. It should not perform the full LiteLLM migration or choose and deploy a production database.

Update the repository charter

  • Rewrite README, AGENTS, ROADMAP, and FEATURES framing so the reliability proxy is the first collection tap, not the final product identity.
  • Preserve an honest inventory of what is implemented today.
  • Make clear that the heavy processing plane remains off the latency-sensitive model request path.
  • Remove instructions that prohibit later capability work merely because phase 1 was once locked.

Add an architecture v2 document

Define:

  • LiteLLM as the provider, routing, retry, fallback, key, budget, and cost layer.
  • Agent Proxy hot-path responsibilities: identity, policy, correlation, context safety, cheap structural detectors, and event emission.
  • Agent Proxy cold-path responsibilities: ingest, normalization, durable raw retention, replay, trajectory assembly, evaluation joins, and dataset materialization.
  • Ward responsibilities for authorization, execution, lifecycle, recovery, and governance.
  • SigNoz and OTLP as operational evidence surfaces rather than the only training-data store.
  • Evaluator inputs and outputs.
  • Content privacy, redaction, retention, and access tiers.

Draft trajectory contract v1

Document a versioned event envelope covering:

  • Stable event id and schema version.
  • Occurred and observed timestamps.
  • Trace, span, Ward run, episode, agent session, request, repo, issue, and workflow correlation.
  • Actor, proposed action, policy decision, execution, observation, state change, evaluation, human intervention, and artifact event types.
  • Model, provider, token, latency, cost, retry, fallback, and finish information.
  • Before and after state references rather than forcing large state into every event.
  • Idempotency, duplicate delivery, partial trajectories, late events, and replay.
  • Provenance and content hashes for derived datasets.
  • Redaction and opt-in body capture.

Use standard OpenTelemetry and GenAI attributes where they fit. Keep custom agentproxy.* fields for the domain-specific joins that standards do not express.

Inventory current code

Classify the major components as:

  • retain unchanged,
  • retain as Agent Proxy-specific behavior,
  • migrate onto LiteLLM,
  • move off the hot path,
  • or retire after parity.

At minimum cover models.py, upstream.py, queue.py, resilience.py, analysis.py, obs.py, and skill_use.py.

File fresh implementation issues

Create new, dependency-ordered Agent Proxy issues for:

  1. LiteLLM standalone-versus-SDK parity spike.
  2. Trajectory schema package and validation fixtures.
  3. Append-only ingestion plus replayable raw retention.
  4. Episode and trajectory materialization.
  5. Evaluation and annotation records.
  6. Versioned SFT, preference, verifier, reward, and held-out-evaluation exports.
  7. Operational queries, Ward dossier inputs, and harness-fit comparison views.

Do not reopen old issues for any of these tracks. Cross-link historical issues only as evidence.

Constraints

  • Do not delete current reliability behavior until the LiteLLM parity issue proves replacement behavior.
  • Do not make SigNoz the sole durable trajectory store.
  • Do not put heavy ML or data processing in the request hot path.
  • Do not make Agent Proxy an execution authority. Ward retains that boundary.
  • Do not claim unimplemented v2 components as landed in docs/FEATURES.md.

Acceptance criteria

  • A new contributor can explain the complete stack and the ownership boundary without reading closed historical issues.
  • The repository docs and coilyco-bridge/agentic-os-hardware#36 point to the same architecture.
  • Trajectory contract v1 is concrete enough for independent producers and consumers to implement against.
  • Every current major module has a documented migration disposition.
  • Fresh downstream issues exist with explicit dependencies and acceptance criteria.
  • No historical issue is reopened.
## Goal Reset the Agent Proxy repository charter around the direction captured in coilyco-bridge/agentic-os-hardware#36: > Agent Proxy is the observation, trajectory collection, and data-processing plane for the agentic operations stack. LiteLLM supplies the commodity inference gateway beneath it. This is a new work item. Historical issues remain closed evidence and must not be reopened or repurposed. ## Why now The repository has delivered a useful reliability proxy, but its current identity and roadmap lag the later product thesis: - There is no LiteLLM dependency. - `models.py`, `upstream.py`, `queue.py`, and `resilience.py` implement substantial commodity gateway behavior. - The service remains stateless apart from its in-memory queue. - Ward correlation metadata and skill-use artifact ingestion have landed, but the latter reduces records to logs and a Prometheus counter rather than persisting trajectories. - `docs/ROADMAP.md` still places data persistence in an unspecified future phase. - coilysiren/inbox#233 states the intended role directly: Agent Proxy is the dataset builder. ## Scope This issue is the repository-level architecture and work-graph reset. It should not perform the full LiteLLM migration or choose and deploy a production database. ### Update the repository charter - Rewrite README, AGENTS, ROADMAP, and FEATURES framing so the reliability proxy is the first collection tap, not the final product identity. - Preserve an honest inventory of what is implemented today. - Make clear that the heavy processing plane remains off the latency-sensitive model request path. - Remove instructions that prohibit later capability work merely because phase 1 was once locked. ### Add an architecture v2 document Define: - LiteLLM as the provider, routing, retry, fallback, key, budget, and cost layer. - Agent Proxy hot-path responsibilities: identity, policy, correlation, context safety, cheap structural detectors, and event emission. - Agent Proxy cold-path responsibilities: ingest, normalization, durable raw retention, replay, trajectory assembly, evaluation joins, and dataset materialization. - Ward responsibilities for authorization, execution, lifecycle, recovery, and governance. - SigNoz and OTLP as operational evidence surfaces rather than the only training-data store. - Evaluator inputs and outputs. - Content privacy, redaction, retention, and access tiers. ### Draft trajectory contract v1 Document a versioned event envelope covering: - Stable event id and schema version. - Occurred and observed timestamps. - Trace, span, Ward run, episode, agent session, request, repo, issue, and workflow correlation. - Actor, proposed action, policy decision, execution, observation, state change, evaluation, human intervention, and artifact event types. - Model, provider, token, latency, cost, retry, fallback, and finish information. - Before and after state references rather than forcing large state into every event. - Idempotency, duplicate delivery, partial trajectories, late events, and replay. - Provenance and content hashes for derived datasets. - Redaction and opt-in body capture. Use standard OpenTelemetry and GenAI attributes where they fit. Keep custom `agentproxy.*` fields for the domain-specific joins that standards do not express. ### Inventory current code Classify the major components as: - retain unchanged, - retain as Agent Proxy-specific behavior, - migrate onto LiteLLM, - move off the hot path, - or retire after parity. At minimum cover `models.py`, `upstream.py`, `queue.py`, `resilience.py`, `analysis.py`, `obs.py`, and `skill_use.py`. ### File fresh implementation issues Create new, dependency-ordered Agent Proxy issues for: 1. LiteLLM standalone-versus-SDK parity spike. 2. Trajectory schema package and validation fixtures. 3. Append-only ingestion plus replayable raw retention. 4. Episode and trajectory materialization. 5. Evaluation and annotation records. 6. Versioned SFT, preference, verifier, reward, and held-out-evaluation exports. 7. Operational queries, Ward dossier inputs, and harness-fit comparison views. Do not reopen old issues for any of these tracks. Cross-link historical issues only as evidence. ## Constraints - Do not delete current reliability behavior until the LiteLLM parity issue proves replacement behavior. - Do not make SigNoz the sole durable trajectory store. - Do not put heavy ML or data processing in the request hot path. - Do not make Agent Proxy an execution authority. Ward retains that boundary. - Do not claim unimplemented v2 components as landed in `docs/FEATURES.md`. ## Acceptance criteria - A new contributor can explain the complete stack and the ownership boundary without reading closed historical issues. - The repository docs and coilyco-bridge/agentic-os-hardware#36 point to the same architecture. - Trajectory contract v1 is concrete enough for independent producers and consumers to implement against. - Every current major module has a documented migration disposition. - Fresh downstream issues exist with explicit dependencies and acceptance criteria. - No historical issue is reopened.
coilyco-ops commented 2026-07-23 05:30:22 +00:00
Author
Member
Copy link

Canonical-plan companion: coilyco-bridge/agentic-os-hardware#36. Agent Proxy #40 should keep its architecture v2 and downstream issues aligned with the aosh source-of-truth correction, without reopening historical tickets.

Canonical-plan companion: https://forgejo.coilysiren.me/coilyco-bridge/agentic-os-hardware/issues/36. Agent Proxy #40 should keep its architecture v2 and downstream issues aligned with the aosh source-of-truth correction, without reopening historical tickets.
coilyco-ops commented 2026-07-23 05:37:21 +00:00
Author
Member
Copy link

WARDED_WORKFLOW: done ✅

details

review summary: charter, architecture v2, trajectory contract v1, migration inventory, and dependency-ordered downstream issues #41 through #47 landed on main.
workflow: merge-remote-main; review summary: in-container review gate intentionally skipped by the temporary ward default pending brokered QA.
retrospective: this felt like a clean architectural reset that preserved the working reliability tap while making the next boundaries explicit.
confidence: high.
surprises: the repository type check has pre-existing skill-use typing failures under the freshly resolved Python 3.13 toolchain. Tests, Ruff, Black, daemonless boot probe, and import smoke passed.
follow-ups: execute #41 and #42 in parallel, then follow the documented retention and materialization dependencies.

WARDED_WORKFLOW: done ✅ <details><summary>details</summary> review summary: charter, architecture v2, trajectory contract v1, migration inventory, and dependency-ordered downstream issues #41 through #47 landed on main. workflow: merge-remote-main; review summary: in-container review gate intentionally skipped by the temporary ward default pending brokered QA. retrospective: this felt like a clean architectural reset that preserved the working reliability tap while making the next boundaries explicit. confidence: high. surprises: the repository type check has pre-existing skill-use typing failures under the freshly resolved Python 3.13 toolchain. Tests, Ruff, Black, daemonless boot probe, and import smoke passed. follow-ups: execute #41 and #42 in parallel, then follow the documented retention and materialization dependencies. </details>
coilyco-ops commented 2026-07-23 05:37:27 +00:00
Author
Member
Copy link

WARDED_WORKFLOW: reservation-released

release details

Run finished with WARDED_WORKFLOW: done ✅.

ward container reap released container engineer-codex-agent-proxy-40 (--harness codex): the terminal outcome supersedes the reservation, so a later redispatch no longer needs --override-reservation.

— Codex, via ward agent

<!-- ward-agent-reservation-released --> WARDED_WORKFLOW: reservation-released <details><summary>release details</summary> Run finished with `WARDED_WORKFLOW: done ✅`. `ward container reap` released container `engineer-codex-agent-proxy-40` (`--harness codex`): the terminal outcome supersedes the reservation, so a later redispatch no longer needs `--override-reservation`. </details> <!-- ward-agent-signature --> — Codex, via `ward agent`
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
task/142-single-ledger-read
task/view-rebuild-connection-churn
task/issue-140-sse-chunk-spans
task/served-model-attribute
aos/claude/mg96-fm
configurable-rate-limits
prefer-backend-header
saturation-stickiness
context-budget-per-model
trim-tool-call-groups
saturation-failover
sse-heartbeats
backend-regime-spans
queue-wait-measures-the-wait
cancel-abandoned-work
upstream-error-classification
prompt-cache-accounting
build-publish-image
doc-layout-caps
exception-taxonomy
skill-fit-view
comment-block-discipline
otel-error-span-path-coverage
restore-all-files-precommit-verb
remove-ungrounded-action-claim-check
aos/claude/adopt-precommit
aos/claude/user-message-capture
issue-82
issue-80
recovery/2026-07-27-local-work
ward-salvage/agent-proxy-09877ee0
No results found.
Labels
Clear labels
  
autonomy
async-consult
 A human needs to consult on the issue to upgrade it to headless

  
autonomy
epic
 This issue has many units of sub work - its size makes it meaningfully exclusive with other autonomy types

  
autonomy
headless
 The agent can perform the work on its own

  
autonomy
live-collab
 The agent and the human need to work together in realtime

  
coherence-core
Core review set for the warded control plane coherence milestone. These issues form the release spine; adjacent milestone issues are stretch or supporting work.

  
priority
P0
 priority tier

  
priority
P1
 priority tier

  
priority
P2
 priority tier

  
priority
P3
 priority tier

  
priority
P4
 priority tier

  
qa-fixture
Disposable issue admitted to the bounded Ward QA verification lane.

  
role/ai
requires work from the AI Engineer role

  
role/creator
requires work from Content Creator role

  
role/design
requires work from the design role

  
role/director
requires work from the director role

  
role/engineer
requires work from the engineer role

  
role/exec
requires work from the exec role

  
role/human
requires a person, and specifically not an agent seat

  
role/ops
requires work from the ops role

  
role/qa
requires work from the QA role

No labels
autonomy
async-consult
autonomy
epic
autonomy
headless
autonomy
live-collab
coherence-core
priority
P0
priority
P1
priority
P2
priority
P3
priority
P4
qa-fixture
role/ai
role/creator
role/design
role/director
role/engineer
role/exec
role/human
role/ops
role/qa
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
coilyco-flight-deck/agent-proxy#40
No description provided.