coilyco-flight-deck/agent-proxy

Watch

Emit the user message as its own capture attribute #85

New issue

Closed

opened 2026-08-11 00:02:40 +00:00 by coilyco-ops · 2 comments

coilyco-ops commented

2026-08-11 00:02:40 +00:00

Member

Raised while building a SigNoz log parser over the capture events, in coilyco-bridge/deploy services/signoz-pipelines/pipelines/20-agent-proxy-capture.json.

What works today

model.response.captured is cleanly parseable. A downstream pipeline promotes finish_reason, usage.completion_tokens, usage.prompt_tokens, and choices[0].message.content into their own attributes, because every one of those sits at a fixed path. That turned a JSON read into a glance and made the truncation defect at coilyco-gaming/sirens-echo#86 visible on a dashboard.

What does not

model.request.captured is not parseable the same way. The user's actual message is the last element of messages, whose length varies with the system prompt, conversation history, and any tool-result rounds. In one observed turn the array had grown to about 47k prompt tokens.

SigNoz pipeline processors cannot express this. move and copy take fixed field paths with no array indexing at all, and the add processor's EXPR(...) escape hatch reaches a known index like EXPR(attributes.parsed.messages[0].content) but has no last-element form.

The available workarounds are all bad:

A regex against the raw body works until the message shape changes, and it would silently pick the wrong message whenever a tool round appends after the user turn.
A fixed negative index is not supported.
Reading the whole request.body on a dashboard defeats the purpose, since the field is dominated by the tools array and history.

Ask

Emit the user message as its own attribute on model.request.captured, alongside the existing agentproxy.* fields. Something like agentproxy.user_message.

The proxy already knows which message it is at capture time, so this is cheap where it sits and impossible everywhere downstream.

Note on safety

Whatever this field is called, it carries untrusted end-user text verbatim, so it inherits the same body-safety treatment as the existing captured bodies rather than being treated as a safe label. Flagging so it is a deliberate decision rather than a default.

Raised while building a SigNoz log parser over the capture events, in coilyco-bridge/deploy `services/signoz-pipelines/pipelines/20-agent-proxy-capture.json`. ## What works today `model.response.captured` is cleanly parseable. A downstream pipeline promotes `finish_reason`, `usage.completion_tokens`, `usage.prompt_tokens`, and `choices[0].message.content` into their own attributes, because every one of those sits at a fixed path. That turned a JSON read into a glance and made the truncation defect at coilyco-gaming/sirens-echo#86 visible on a dashboard. ## What does not `model.request.captured` is not parseable the same way. The user's actual message is the last element of `messages`, whose length varies with the system prompt, conversation history, and any tool-result rounds. In one observed turn the array had grown to about 47k prompt tokens. SigNoz pipeline processors cannot express this. `move` and `copy` take fixed field paths with no array indexing at all, and the `add` processor's `EXPR(...)` escape hatch reaches a known index like `EXPR(attributes.parsed.messages[0].content)` but has no last-element form. The available workarounds are all bad: * A regex against the raw body works until the message shape changes, and it would silently pick the wrong message whenever a tool round appends after the user turn. * A fixed negative index is not supported. * Reading the whole `request.body` on a dashboard defeats the purpose, since the field is dominated by the tools array and history. ## Ask Emit the user message as its own attribute on `model.request.captured`, alongside the existing `agentproxy.*` fields. Something like `agentproxy.user_message`. The proxy already knows which message it is at capture time, so this is cheap where it sits and impossible everywhere downstream. ## Note on safety Whatever this field is called, it carries untrusted end-user text verbatim, so it inherits the same body-safety treatment as the existing captured bodies rather than being treated as a safe label. Flagging so it is a deliberate decision rather than a default.

coilyco-ops referenced this issue from a commit

2026-08-11 00:10:00 +00:00

feat: emit the final user message as its own capture attribute

coilyco-ops commented

2026-08-11 00:13:12 +00:00

Author

Member

Implemented on branch aos/claude/user-message-capture, commit 632745c. Needs a pull request opened, which the agent Forgejo surface cannot do.

Compare: https://forgejo.coilysiren.me/coilyco-flight-deck/agent-proxy/compare/main...aos/claude/user-message-capture

What landed

model.request.captured now carries agentproxy.user_message with the verbatim text of the final user turn.

last_user_message() in app/body_capture.py reads the messages list backwards for the last user role. Reading backwards rather than taking the last element is the point: a tool-using turn ends on a tool result, so the last element is usually not the user at all. That case has its own test.

Both content shapes are handled. A plain string returns as-is, and the content-parts form joins the text parts and ignores the rest.

Design decisions worth reviewing

Extraction is total and never raises. The field is a convenience projection beside the complete body, not a selected-field capture mode, so an absent, blank, or unreadable user message omits the field rather than failing a capture that would otherwise have succeeded. This is deliberately weaker than the surrounding module's fail-hard contract, because a request with no user message is legitimate and must not break capture.

The capture schema version stays at 1. Adding an optional field is backward compatible, and nothing in the repository pins the value, so a bump would signal an incompatibility that does not exist.

No truncation. A long user message is emitted whole. The complete body already carries the same text, so truncating would add a lossy field without reducing exposure.

Safety

The field carries untrusted end-user text verbatim, so it inherits the same restricted treatment as the captured bodies it sits beside. It adds no exposure that request.body did not already have, and it is only emitted when capture is enabled.

Verification

ward exec format-check, lint, typecheck, and test all pass. 248 tests, 8 of them new: last-turn-not-first, trailing tool rounds, content-parts joining, an eight-case parametrized none-rather-than-raise sweep, and both emit paths.

Not run: ward exec test-container, which needs docker.

Downstream

The consuming pipeline is already deployed at coilyco-bridge/deploy services/signoz-pipelines/pipelines/20-agent-proxy-capture.json, which currently parses the response side only. Once this ships, the request half can promote agentproxy.user_message the same way and the Sirens Deep console can show the question next to the answer.

Implemented on branch `aos/claude/user-message-capture`, commit `632745c`. Needs a pull request opened, which the agent Forgejo surface cannot do. Compare: https://forgejo.coilysiren.me/coilyco-flight-deck/agent-proxy/compare/main...aos/claude/user-message-capture ## What landed `model.request.captured` now carries `agentproxy.user_message` with the verbatim text of the final user turn. `last_user_message()` in `app/body_capture.py` reads the messages list backwards for the last `user` role. Reading backwards rather than taking the last element is the point: a tool-using turn ends on a tool result, so the last element is usually not the user at all. That case has its own test. Both content shapes are handled. A plain string returns as-is, and the content-parts form joins the `text` parts and ignores the rest. ## Design decisions worth reviewing **Extraction is total and never raises.** The field is a convenience projection beside the complete body, not a selected-field capture mode, so an absent, blank, or unreadable user message omits the field rather than failing a capture that would otherwise have succeeded. This is deliberately weaker than the surrounding module's fail-hard contract, because a request with no user message is legitimate and must not break capture. **The capture schema version stays at 1.** Adding an optional field is backward compatible, and nothing in the repository pins the value, so a bump would signal an incompatibility that does not exist. **No truncation.** A long user message is emitted whole. The complete body already carries the same text, so truncating would add a lossy field without reducing exposure. ## Safety The field carries untrusted end-user text verbatim, so it inherits the same restricted treatment as the captured bodies it sits beside. It adds no exposure that `request.body` did not already have, and it is only emitted when capture is enabled. ## Verification `ward exec format-check`, `lint`, `typecheck`, and `test` all pass. 248 tests, 8 of them new: last-turn-not-first, trailing tool rounds, content-parts joining, an eight-case parametrized none-rather-than-raise sweep, and both emit paths. Not run: `ward exec test-container`, which needs docker. ## Downstream The consuming pipeline is already deployed at coilyco-bridge/deploy `services/signoz-pipelines/pipelines/20-agent-proxy-capture.json`, which currently parses the response side only. Once this ships, the request half can promote `agentproxy.user_message` the same way and the Sirens Deep console can show the question next to the answer.

coilysiren referenced this issue from a pull request that will close it,

2026-08-11 00:14:11 +00:00

feat: emit the final user message as its own capture attribute #86

coilyco-ops commented

2026-08-11 00:32:48 +00:00

Author

Member

Extended the same branch with the response half, commit ff28539. The open PR picks it up.

Why the scope grew

The first commit derived only the input message, because that was the half a log pipeline provably cannot reach. The response fields sit at fixed paths, so a downstream pipeline handles them today and I left them there.

That reasoning was right about capability and wrong about ownership. It left turn-shape fields split across two repositories, and it meant only the SigNoz ingest path saw them. The trajectory store, evaluation joins, and any other capture consumer got the raw body and nothing else.

What the response side adds

model.response.captured now carries agentproxy.finish_reason, agentproxy.assistant_message, agentproxy.completion_tokens, and agentproxy.prompt_tokens, from the first choice and the usage block. The complete body still carries everything, further choices and reasoning content included.

A truncated completion is now legible from the projection alone: finish_reason of length, completion_tokens at the request's cap, and no assistant message. That is the coilyco-gaming/sirens-echo#86 signature without opening a body.

Projection is total on both sides

An incomplete response keeps its closed-set reason and still projects whatever it does carry. A blank assistant message omits its field rather than emitting an empty string, which is what makes the truncation case read cleanly. Booleans are rejected where an integer token count is expected, since bool is an int subclass in Python.

Verification

ward exec format-check, lint, typecheck, and test all pass. 260 tests, up from 248, with 12 new on the response side covering the projection, the budget-exhausted shape, an eight-case omit-rather-than-raise sweep, and both the complete and incomplete emit paths.

Sequencing note for whoever merges

The deploy-side parser at coilyco-bridge/deploy services/signoz-pipelines/pipelines/20-agent-proxy-capture.json currently derives the response fields itself and is feeding a live dashboard. Do not thin it out until this ships and ser8 has rolled, or the Sirens Deep console loses those columns in the gap. Once the new image is serving, that parser can drop its extraction and the pipeline reduces to promoting the attributes Agent Proxy already emits.

Extended the same branch with the response half, commit `ff28539`. The open PR picks it up. ## Why the scope grew The first commit derived only the input message, because that was the half a log pipeline provably cannot reach. The response fields sit at fixed paths, so a downstream pipeline handles them today and I left them there. That reasoning was right about capability and wrong about ownership. It left turn-shape fields split across two repositories, and it meant only the SigNoz ingest path saw them. The trajectory store, evaluation joins, and any other capture consumer got the raw body and nothing else. ## What the response side adds `model.response.captured` now carries `agentproxy.finish_reason`, `agentproxy.assistant_message`, `agentproxy.completion_tokens`, and `agentproxy.prompt_tokens`, from the first choice and the usage block. The complete body still carries everything, further choices and reasoning content included. A truncated completion is now legible from the projection alone: `finish_reason` of `length`, `completion_tokens` at the request's cap, and no assistant message. That is the coilyco-gaming/sirens-echo#86 signature without opening a body. ## Projection is total on both sides An incomplete response keeps its closed-set reason and still projects whatever it does carry. A blank assistant message omits its field rather than emitting an empty string, which is what makes the truncation case read cleanly. Booleans are rejected where an integer token count is expected, since `bool` is an `int` subclass in Python. ## Verification `ward exec format-check`, `lint`, `typecheck`, and `test` all pass. 260 tests, up from 248, with 12 new on the response side covering the projection, the budget-exhausted shape, an eight-case omit-rather-than-raise sweep, and both the complete and incomplete emit paths. ## Sequencing note for whoever merges The deploy-side parser at coilyco-bridge/deploy `services/signoz-pipelines/pipelines/20-agent-proxy-capture.json` currently derives the response fields itself and is feeding a live dashboard. **Do not thin it out until this ships and ser8 has rolled**, or the Sirens Deep console loses those columns in the gap. Once the new image is serving, that parser can drop its extraction and the pipeline reduces to promoting the attributes Agent Proxy already emits.

coilysiren closed this issue

2026-08-11 00:34:14 +00:00

coilyco-ops referenced this issue from coilyco-gaming/sirens-echo

2026-08-11 08:42:42 +00:00

The final user message is a composed blob rather than the user's message #104