Debug host-local agent-proxy OTLP export to SigNoz #439

New issue

Closed

opened 2026-07-02 10:59:52 +00:00 by coilyco-ops · 0 comments

coilyco-ops commented

2026-07-02 10:59:52 +00:00

Owner

Summary

Local runtime verification on kais-macbook-pro-2 shows agent-proxy is up and serving, but its OTLP traces are not arriving in the host-local SigNoz backend.

Evidence

ward exec ansible-sync local=1 tags=signoz,agent-proxy converged cleanly on 2026-07-02.
curl -fsS http://127.0.0.1:8082/healthz returned {"status":"ok"}.
curl -fsS http://127.0.0.1:8082/v1/models returned the expected logical model list.
~/.config/agent-proxy/.env contains PROXY_OTEL_EXPORTER_OTLP_ENDPOINT=http://host.docker.internal:4318.
Emitting a span from inside the running agent-proxy container through the app's own app.obs OTel setup produced repeated RemoteDisconnected('Remote end closed connection without response') export failures.
ClickHouse still shows no rows in signoz_traces.signoz_index_v3 (max(timestamp) is the zero epoch).

Notes

Reproduced both with the app's configured exporter path and with a manual OTLP HTTP exporter pointed at http://host.docker.internal:4318/v1/traces.
The local Forgejo CI for commit 4043116 is green, so this is a runtime-path bug, not a repo validation failure.

## Summary Local runtime verification on `kais-macbook-pro-2` shows `agent-proxy` is up and serving, but its OTLP traces are not arriving in the host-local SigNoz backend. ## Evidence - `ward exec ansible-sync local=1 tags=signoz,agent-proxy` converged cleanly on 2026-07-02. - `curl -fsS http://127.0.0.1:8082/healthz` returned `{"status":"ok"}`. - `curl -fsS http://127.0.0.1:8082/v1/models` returned the expected logical model list. - `~/.config/agent-proxy/.env` contains `PROXY_OTEL_EXPORTER_OTLP_ENDPOINT=http://host.docker.internal:4318`. - Emitting a span from inside the running `agent-proxy` container through the app's own `app.obs` OTel setup produced repeated `RemoteDisconnected('Remote end closed connection without response')` export failures. - ClickHouse still shows no rows in `signoz_traces.signoz_index_v3` (`max(timestamp)` is the zero epoch). ## Notes - Reproduced both with the app's configured exporter path and with a manual OTLP HTTP exporter pointed at `http://host.docker.internal:4318/v1/traces`. - The local Forgejo CI for commit `4043116` is green, so this is a runtime-path bug, not a repo validation failure.