feat(mirror): send the tool-call trajectory to Temporal, metadata only #888

Merged

coilysiren merged 1 commit from aos/claude/hs68-temporal-mirror into main

2026-08-17 04:43:35 +00:00

coilyco-ops commented

2026-08-17 04:37:33 +00:00

Member

Mirroring, not orchestration. Temporal observes the turn and never runs it, so nothing here sits in the control path and nothing compounds with agent-proxy's fallback.

The payload is a struct, and that is the safety property

ToolCallRecord has five named fields: server, tool, outcome, elapsed millis, trace id.

It is deliberately not the span's attribute slice. The issue named the exact failure and it is worth restating, because everything else in this PR is downstream of it: StartSpan is a variadic passthrough, so copying from it would mirror whatever any caller passed, sight unseen, forever. Someone adds attribute.String("message.text", ...) to a span one day and Discord member content starts flowing to a third-party SaaS, with no change at the mirror and nothing to notice it.

Widening what leaves now means adding a field to that struct, which is a disclosure decision someone makes on purpose. TestTheMirroredPayloadCarriesNothingButMetadata enumerates the fields and fails when the set changes, rather than sampling for known-bad strings.

The trace id is the only field outside RecordToolCall's own triple. It names no person, and a member is already handed one in a failure notice.

Tool calls, not spans

Only RecordToolCall mirrors. TestOnlyToolCallsAreMirrored runs three StartSpan calls, a RecordTurn, a RecordModelCall and a RecordFailure alongside two tool calls, and asserts two records.

SignalWithStartWorkflow keyed on sirens-deep-trajectory-<trace id> is one action whether the turn's workflow exists yet or not, so volume tracks tool calls. A turn's calls land in one ordered trajectory instead of one workflow each, which is also what makes it a trajectory rather than a pile.

Nothing about it can reach a turn

Four ways this could have become a new way for turns to die, and what each does instead:

failure	behaviour
backend slow or wedged	non-blocking send, single worker, hard timeout
backend erroring	error swallowed, drop counted
client panicking	recovered, drop counted
queue full	drops rather than blocks

Each increments sirens_echo.mirror.drops, so an outage is a number rather than the silence #137 and #190 describe. The delivery context is detached from the turn, so a finished turn does not cancel its own audit record.

TestAStalledMirrorNeverBlocksTheTurn sends 500 records with the only worker wedged and asserts the sender returns.

Configuration

Off unless a deployment supplies all of SIRENS_ECHO_TEMPORAL_HOST, _NAMESPACE, and _TASK_QUEUE.

A half-filled connection fails at boot. A typo that quietly turned the mirror off would be the same silent failure the drop counter exists to prevent. A dial failure is deliberately different and only logs: Temporal being unreachable must never stop this service answering.

Deep only, and the owl-glass surface first, are deployment choices made by which lane sets the variables. That is coilyco-bridge/deploy's, per the issue's scope boundary. The credential is #444.

Acceptance

criterion	how
a record carrying server, tool, outcome, timing	`TestAToolCallIsMirroredWithItsMetadata`, `TestTheSignalCarriesTheRecordUnchanged`
no message text, arguments, results, or user ids	`TestTheMirroredPayloadCarriesNothingButMetadata` - field enumeration plus an encoded-form check
an outage leaves turns unaffected, drops observable	`TestAStalledMirrorNeverBlocksTheTurn`, `TestAFailingMirrorIsCountedAndNeverReturned`, `TestAPanickingMirrorIsContained`
roughly one action per tool call	`TestEachToolCallIsOneSignalKeyedOnTheTurn`, `TestOnlyToolCallsAreMirrored`
a content-bearing span attribute changes nothing	`TestAContentBearingSpanAttributeDoesNotReachTheMirror` - puts `message.text` and `mcp.tool.arguments` on the span, asserts neither reaches the payload

Two things worth a reviewer's eye

go.temporal.io/sdk moves the go directive from 1.25.0 to 1.25.4, because the SDK's own go.mod requires it. No GOTOOLCHAIN pin exists in CI, so Go's toolchain switching handles it, and CI is the check.

Nothing here has talked to Temporal Cloud. The credential is #444 and the deployment wiring is deploy's, so every test substitutes the one client call behind an interface. The transport is the part still unproven, and the first real connection will be the test of it.

just gate passes.

closes #887

Mirroring, not orchestration. Temporal observes the turn and never runs it, so nothing here sits in the control path and nothing compounds with agent-proxy's fallback. ## The payload is a struct, and that is the safety property `ToolCallRecord` has five named fields: server, tool, outcome, elapsed millis, trace id. **It is deliberately not the span's attribute slice.** The issue named the exact failure and it is worth restating, because everything else in this PR is downstream of it: `StartSpan` is a variadic passthrough, so copying from it would mirror whatever any caller passed, sight unseen, forever. Someone adds `attribute.String("message.text", ...)` to a span one day and Discord member content starts flowing to a third-party SaaS, with no change at the mirror and nothing to notice it. Widening what leaves now means adding a field to that struct, which is a disclosure decision someone makes on purpose. `TestTheMirroredPayloadCarriesNothingButMetadata` enumerates the fields and fails when the set changes, rather than sampling for known-bad strings. The trace id is the only field outside `RecordToolCall`'s own triple. It names no person, and a member is already handed one in a failure notice. ## Tool calls, not spans Only `RecordToolCall` mirrors. `TestOnlyToolCallsAreMirrored` runs three `StartSpan` calls, a `RecordTurn`, a `RecordModelCall` and a `RecordFailure` alongside two tool calls, and asserts two records. `SignalWithStartWorkflow` keyed on `sirens-deep-trajectory-<trace id>` is one action whether the turn's workflow exists yet or not, so volume tracks tool calls. A turn's calls land in one ordered trajectory instead of one workflow each, which is also what makes it a trajectory rather than a pile. ## Nothing about it can reach a turn Four ways this could have become a new way for turns to die, and what each does instead: | failure | behaviour | | --- | --- | | backend slow or wedged | non-blocking send, single worker, hard timeout | | backend erroring | error swallowed, drop counted | | client panicking | recovered, drop counted | | queue full | drops rather than blocks | Each increments `sirens_echo.mirror.drops`, so an outage is a number rather than the silence #137 and #190 describe. The delivery context is detached from the turn, so a finished turn does not cancel its own audit record. `TestAStalledMirrorNeverBlocksTheTurn` sends 500 records with the only worker wedged and asserts the sender returns. ## Configuration Off unless a deployment supplies all of `SIRENS_ECHO_TEMPORAL_HOST`, `_NAMESPACE`, and `_TASK_QUEUE`. **A half-filled connection fails at boot.** A typo that quietly turned the mirror off would be the same silent failure the drop counter exists to prevent. A dial failure is deliberately different and only logs: Temporal being unreachable must never stop this service answering. Deep only, and the owl-glass surface first, are deployment choices made by which lane sets the variables. That is `coilyco-bridge/deploy`'s, per the issue's scope boundary. The credential is #444. ## Acceptance | criterion | how | | --- | --- | | a record carrying server, tool, outcome, timing | `TestAToolCallIsMirroredWithItsMetadata`, `TestTheSignalCarriesTheRecordUnchanged` | | no message text, arguments, results, or user ids | `TestTheMirroredPayloadCarriesNothingButMetadata` - field enumeration plus an encoded-form check | | an outage leaves turns unaffected, drops observable | `TestAStalledMirrorNeverBlocksTheTurn`, `TestAFailingMirrorIsCountedAndNeverReturned`, `TestAPanickingMirrorIsContained` | | roughly one action per tool call | `TestEachToolCallIsOneSignalKeyedOnTheTurn`, `TestOnlyToolCallsAreMirrored` | | a content-bearing span attribute changes nothing | `TestAContentBearingSpanAttributeDoesNotReachTheMirror` - puts `message.text` and `mcp.tool.arguments` on the span, asserts neither reaches the payload | ## Two things worth a reviewer's eye **`go.temporal.io/sdk` moves the go directive from 1.25.0 to 1.25.4**, because the SDK's own `go.mod` requires it. No `GOTOOLCHAIN` pin exists in CI, so Go's toolchain switching handles it, and CI is the check. **Nothing here has talked to Temporal Cloud.** The credential is #444 and the deployment wiring is deploy's, so every test substitutes the one client call behind an interface. The transport is the part still unproven, and the first real connection will be the test of it. `just gate` passes. closes #887

coilyco-ops added 1 commit

2026-08-17 04:37:33 +00:00

feat(mirror): send the tool-call trajectory to Temporal, metadata only

ci / test (pull_request) Successful in 51s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

ci / image-build (pull_request) Successful in 33s

Details

d5f07fe2ac

Deep now emits a durable record of what it called into Temporal Cloud. Temporal
observes the turn rather than running it, so nothing here is in the control path
and nothing compounds with agent-proxy's fallback.

The payload is a five-field struct, not the span's attribute slice. StartSpan is
a variadic passthrough, so copying from it would mirror whatever any caller
passed, sight unseen, forever: someone adds attribute.String("message.text", ..)
to a span one day and member content starts flowing to a third-party SaaS with
no change at the mirror and nothing to notice it. Widening what leaves now means
adding a field, which a test enumerates and fails on.

Only RecordToolCall mirrors. StartSpan runs many times per turn and does not,
because hooking span-start points a firehose at a service that bills per action.
SignalWithStart keyed on the trace id is one action per call and gives a turn
one ordered trajectory rather than one workflow per call.

Nothing about it can reach a turn. The send is a non-blocking channel write, one
worker owns the only call, the context is detached from the turn and hard
bounded, errors are swallowed and a panicking client is recovered. Every one of
those paths counts sirens_echo.mirror.drops, so an outage is a number rather
than the silence of #137 and #190.

Off unless a deployment supplies host, namespace, and task queue. A half-filled
connection fails at boot, because a typo that quietly disabled the mirror is the
same silent failure the drop counter exists to prevent. A dial failure only
logs: Temporal being down must never stop this service answering.

The worker that runs ToolTrajectoryWorkflow is the deployment's, the credential
is #444, and which lane sets the variables is what keeps this Deep only.

closes #887

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>