Ingest ward correlation metadata into agent-proxy traces and logs #38

New issue

Closed

opened 2026-07-09 19:06:12 +00:00 by coilyco-ops · 2 comments

coilyco-ops commented

2026-07-09 19:06:12 +00:00

Member

Goal

Let agent-proxy correlate each LLM request with the ward agent / warded run that produced it, so SigNoz can join proxy traces with ward-run logs.

Current state

app/main.py reads x-request-id and stores it as agentproxy.request_id in RequestTraceContext. That is useful, but it does not preserve the ward run envelope: repo, issue, harness, role, workflow, context level, ward version, or harness-native session ids.

Proposed ingest contract

Accept metadata from headers first, with optional OpenAI body metadata as fallback when a client supports body metadata but not custom headers. Suggested fields:

x-request-id -> agentproxy.request_id
x-ward-run-id -> ward.run_id
x-ward-container-name -> ward.container_name
x-ward-role -> ward.role
x-ward-harness -> ward.harness
x-ward-target-repo -> ward.target_repo
x-ward-issue-ref -> ward.issue_ref
x-ward-workflow -> ward.workflow
x-ward-context-level -> ward.context_level
x-ward-version -> ward.version
x-agent-session-id -> agent.session_id for harness ids such as CODEX_THREAD_ID

Behavior

Add these fields to RequestTraceContext.extra for /v1/chat/completions, /v1/completions, and streaming paths.
Include them in structured logs through request_log_fields.
Add them as span attributes on root request spans and child queue/resilience/upstream spans via the existing trace context propagation.
Keep Prometheus labels low-cardinality. Do not add run id, issue, or container labels to existing histograms/counters.
Add tests for header parsing, body metadata fallback, and precedence.
Update docs/proxy.md and docs/FEATURES.md.

Acceptance

A test request with ward headers produces SigNoz spans/logs queryable by ward.run_id, ward.target_repo, ward.issue_ref, ward.harness, and agentproxy.request_id, while /metrics cardinality remains bounded.

Related: coilyco-flight-deck/ward#861.

## Goal Let agent-proxy correlate each LLM request with the `ward agent` / `warded` run that produced it, so SigNoz can join proxy traces with ward-run logs. ## Current state `app/main.py` reads `x-request-id` and stores it as `agentproxy.request_id` in `RequestTraceContext`. That is useful, but it does not preserve the ward run envelope: repo, issue, harness, role, workflow, context level, ward version, or harness-native session ids. ## Proposed ingest contract Accept metadata from headers first, with optional OpenAI body `metadata` as fallback when a client supports body metadata but not custom headers. Suggested fields: * `x-request-id` -> `agentproxy.request_id` * `x-ward-run-id` -> `ward.run_id` * `x-ward-container-name` -> `ward.container_name` * `x-ward-role` -> `ward.role` * `x-ward-harness` -> `ward.harness` * `x-ward-target-repo` -> `ward.target_repo` * `x-ward-issue-ref` -> `ward.issue_ref` * `x-ward-workflow` -> `ward.workflow` * `x-ward-context-level` -> `ward.context_level` * `x-ward-version` -> `ward.version` * `x-agent-session-id` -> `agent.session_id` for harness ids such as `CODEX_THREAD_ID` ## Behavior * Add these fields to `RequestTraceContext.extra` for `/v1/chat/completions`, `/v1/completions`, and streaming paths. * Include them in structured logs through `request_log_fields`. * Add them as span attributes on root request spans and child queue/resilience/upstream spans via the existing trace context propagation. * Keep Prometheus labels low-cardinality. Do not add run id, issue, or container labels to existing histograms/counters. * Add tests for header parsing, body metadata fallback, and precedence. * Update `docs/proxy.md` and `docs/FEATURES.md`. ## Acceptance A test request with ward headers produces SigNoz spans/logs queryable by `ward.run_id`, `ward.target_repo`, `ward.issue_ref`, `ward.harness`, and `agentproxy.request_id`, while `/metrics` cardinality remains bounded. Related: coilyco-flight-deck/ward#861.

coilyco-ops commented

2026-07-09 19:31:08 +00:00

Author

Member

WARD-RESERVATION: held 🔒

reservation details

Holder: container engineer-codex-agent-proxy-38 on host kais-macbook-pro-2.local.

Reserved by ward agent --harness codex (reserved 2026-07-09T19:31:06Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (1h TTL). --force overrides.

Do not comment on or edit this issue to steer the run while it is reserved. The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a new issue, dispatched fresh. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).

run seed context — what this run is carrying (ward#609)

Resolved: coilyco-flight-deck/agent-proxy#38 · branch issue-38 · harness codex · workflow direct-to-main
Run: engineer-codex-agent-proxy-38 · ward v0.504.0 · dispatched 2026-07-09T19:31:06Z
Comment thread: 0 included in the pre-flight read, 0 stripped (ward's own automated comments).

Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.504.0).

— Codex, via ward agent

WARD-RESERVATION: held 🔒 <details><summary>reservation details</summary> Holder: container `engineer-codex-agent-proxy-38` on host `kais-macbook-pro-2.local`. Reserved by `ward agent --harness codex` (reserved 2026-07-09T19:31:06Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (1h TTL). `--force` overrides. **Do not comment on or edit this issue to steer the run while it is reserved.** The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a **new issue, dispatched fresh**. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494). <details><summary>run seed context — what this run is carrying (ward#609)</summary> - **Resolved:** `coilyco-flight-deck/agent-proxy#38` · branch `issue-38` · harness `codex` · workflow `direct-to-main` - **Run:** `engineer-codex-agent-proxy-38` · ward `v0.504.0` · dispatched `2026-07-09T19:31:06Z` - **Comment thread:** 0 included in the pre-flight read, 0 stripped (ward's own automated comments). Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.504.0). </details> </details>  — Codex, via `ward agent`

coilyco-ops referenced this issue from a commit

2026-07-09 19:39:03 +00:00

feat: ingest ward correlation metadata

coilyco-ops closed this issue

2026-07-09 19:39:04 +00:00

coilyco-ops commented

2026-07-09 19:40:02 +00:00

Author

Member

WARD-OUTCOME: done ✅

details

workflow: direct-to-main; review summary: review skipped by ~/.ward/config.yaml default
felt: straightforward. wired metadata ingestion through route context, spans, logs, and streaming path.
confidence: high
surprises: none
follow-ups: none

WARD-OUTCOME: done ✅ <details><summary>details</summary> workflow: direct-to-main; review summary: review skipped by ~/.ward/config.yaml default felt: straightforward. wired metadata ingestion through route context, spans, logs, and streaming path. confidence: high surprises: none follow-ups: none </details>