coilyco-flight-deck/infrastructure

Fork 0

agent-proxy: container binds PROXY_PORT=8082 but compose maps to container :8080 - proxy never serves on 127.0.0.1:8082 #430

New issue

Closed

opened 2026-07-02 07:55:36 +00:00 by coilysiren · 3 comments

coilysiren commented

2026-07-02 07:55:36 +00:00

Owner

Problem

The agent-proxy role converges the container but it does not serve on 127.0.0.1:8082 - the validation curls in docs/agent-proxy.md (/v1/healthz, /v1/models) fail with a connection reset even though docker ps shows the container up.

Root cause - `PROXY_PORT` is overloaded

The agent-proxy server binds to settings.proxy_port (coilyco-flight-deck/agent-proxy app/__init__.py:59, app/config.py:54), fed by the PROXY_PORT env var, default 8080. The Dockerfile EXPOSEs 8080 and the compose mapping hardcodes the container side to 8080.

But ansible/roles/agent-proxy/tasks/main.yml passes PROXY_PORT: "{{ agent_proxy_port }}" (= 8082, the default) into the container environment. So:

the app inside the container binds 0.0.0.0:8082
files/compose.yaml maps 127.0.0.1:${PROXY_PORT:-8082}:8080 -> host 8082 to container 8080
nothing listens on container 8080, so host 8082 accept-then-resets

PROXY_PORT is doing two jobs at once - the host publish port and the container's internal listen port - and only the host side received 8082.

Fix

Separate the two. The container should always listen on a fixed internal 8080 (matching the Dockerfile EXPOSE and the app default) and only the published host port should vary with agent_proxy_port. Concretely, in this repo (no agent-proxy repo change needed):

files/compose.yaml: publish 127.0.0.1:${AGENT_PROXY_PUBLISH_PORT:-8082}:8080 and pin the container env PROXY_PORT: "8080" (leave PROXY_HOST as-is at 0.0.0.0 so Docker forwarding can reach it).
tasks/main.yml: stop pushing agent_proxy_port into the container's PROXY_PORT; pass it as the publish-port var instead. Keep PROXY_HOST: 0.0.0.0 and PROXY_TOWER_BASE_URL unchanged.

Pick whatever var names read cleanly - the invariant is: published host port = agent_proxy_port, container internal port = fixed 8080, and the two ends of the mapping agree with where the app actually binds.

Acceptance

After ward exec ansible-sync on a mac, curl -fsS http://127.0.0.1:8082/v1/healthz and /v1/models succeed (tower reachable).
docker logs agent-proxy shows the app binding the same container port the mapping targets.
Setting a non-default agent_proxy_port still works end to end (publish port changes, container stays on 8080).

Refs

Introduced in 08f117f (closes #425).
docs/agent-proxy.md validation steps are the reproduction.

## Problem The `agent-proxy` role converges the container but it does not serve on `127.0.0.1:8082` - the validation curls in `docs/agent-proxy.md` (`/v1/healthz`, `/v1/models`) fail with a connection reset even though `docker ps` shows the container up. ## Root cause - `PROXY_PORT` is overloaded The agent-proxy server binds to `settings.proxy_port` (`coilyco-flight-deck/agent-proxy` `app/__init__.py:59`, `app/config.py:54`), fed by the `PROXY_PORT` env var, default `8080`. The Dockerfile `EXPOSE`s `8080` and the compose mapping hardcodes the container side to `8080`. But `ansible/roles/agent-proxy/tasks/main.yml` passes `PROXY_PORT: "{{ agent_proxy_port }}"` (= `8082`, the default) into the container environment. So: - the app inside the container binds `0.0.0.0:8082` - `files/compose.yaml` maps `127.0.0.1:${PROXY_PORT:-8082}:8080` -> host `8082` to container `8080` - nothing listens on container `8080`, so host `8082` accept-then-resets `PROXY_PORT` is doing two jobs at once - the host publish port and the container's internal listen port - and only the host side received `8082`. ## Fix Separate the two. The container should always listen on a fixed internal `8080` (matching the Dockerfile `EXPOSE` and the app default) and only the published host port should vary with `agent_proxy_port`. Concretely, in this repo (no agent-proxy repo change needed): - `files/compose.yaml`: publish `127.0.0.1:${AGENT_PROXY_PUBLISH_PORT:-8082}:8080` and pin the container env `PROXY_PORT: "8080"` (leave `PROXY_HOST` as-is at `0.0.0.0` so Docker forwarding can reach it). - `tasks/main.yml`: stop pushing `agent_proxy_port` into the container's `PROXY_PORT`; pass it as the publish-port var instead. Keep `PROXY_HOST: 0.0.0.0` and `PROXY_TOWER_BASE_URL` unchanged. Pick whatever var names read cleanly - the invariant is: **published host port = `agent_proxy_port`, container internal port = fixed 8080, and the two ends of the mapping agree with where the app actually binds.** ## Acceptance - After `ward exec ansible-sync` on a mac, `curl -fsS http://127.0.0.1:8082/v1/healthz` and `/v1/models` succeed (tower reachable). - `docker logs agent-proxy` shows the app binding the same container port the mapping targets. - Setting a non-default `agent_proxy_port` still works end to end (publish port changes, container stays on 8080). ## Refs - Introduced in `08f117f` (closes #425). - `docs/agent-proxy.md` validation steps are the reproduction.

coilyco-ops commented

2026-07-02 07:55:46 +00:00

Member

🔒 Reserved by ward agent --driver claude — container engineer-claude-infrastructure-430 on host kais-macbook-pro-2.local is carrying this issue (reserved 2026-07-02T07:55:43Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); --force overrides.

— Claude (she/her), via ward agent

🔒 Reserved by `ward agent --driver claude` — container `engineer-claude-infrastructure-430` on host `kais-macbook-pro-2.local` is carrying this issue (reserved 2026-07-02T07:55:43Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); `--force` overrides.  — Claude (she/her), via `ward agent`

coilysiren referenced this issue from a commit

2026-07-02 07:58:30 +00:00

fix(agent-proxy): split published host port from fixed container :8080

coilysiren closed this issue

2026-07-02 07:58:30 +00:00

coilysiren referenced this issue from a commit

2026-07-02 07:58:31 +00:00

Merge issue-430: fix agent-proxy port split (closes #430)

coilysiren commented

2026-07-02 07:59:39 +00:00

Author

Owner

WARD-OUTCOME: done - compose publishes AGENT_PROXY_PUBLISH_PORT->:8080, container PROXY_PORT pinned to 8080, tasks/main.yml passes agent_proxy_port as the publish var

This one was a clean, well-scoped fix - the issue body basically wrote the diff, and the root cause (PROXY_PORT wearing two hats) was exactly as described. The actual code change was three lines across two files.

What fought back wasn't the fix, it was the toolchain: the code-comments hook rejects any YAML comment below the top header block, so my inline "why" comments had to move up into the file header. And the pylint pre-commit hook couldn't run because the container's UV_PYTHON_INSTALL_DIR (/opt/uv/python) is read-only and the project needs Python 3.13 - I redirected uv's install dir to $HOME/.uv-python so the hook could download and pass (10.00/10). Worth flagging as a container-env rough edge if other runs hit the same wall.

Confidence is high on the mapping being correct-by-construction (both ends now agree on :8080, publish port varies), but I could not run the live acceptance curls - no mac / running container / tower reachable from here, and the SSM api-token wasn't available to check Forgejo CI. So the end-to-end /v1/healthz + /v1/models verification still wants a real ward exec ansible-sync on a mac to close out. No follow-ups needed beyond that manual confirmation.

WARD-OUTCOME: done - compose publishes AGENT_PROXY_PUBLISH_PORT->:8080, container PROXY_PORT pinned to 8080, tasks/main.yml passes agent_proxy_port as the publish var This one was a clean, well-scoped fix - the issue body basically wrote the diff, and the root cause (PROXY_PORT wearing two hats) was exactly as described. The actual code change was three lines across two files. What fought back wasn't the fix, it was the toolchain: the `code-comments` hook rejects any YAML comment below the top header block, so my inline "why" comments had to move up into the file header. And the `pylint` pre-commit hook couldn't run because the container's `UV_PYTHON_INSTALL_DIR` (`/opt/uv/python`) is read-only and the project needs Python 3.13 - I redirected uv's install dir to `$HOME/.uv-python` so the hook could download and pass (10.00/10). Worth flagging as a container-env rough edge if other runs hit the same wall. Confidence is high on the mapping being correct-by-construction (both ends now agree on :8080, publish port varies), but I could not run the live acceptance curls - no mac / running container / tower reachable from here, and the SSM api-token wasn't available to check Forgejo CI. So the end-to-end `/v1/healthz` + `/v1/models` verification still wants a real `ward exec ansible-sync` on a mac to close out. No follow-ups needed beyond that manual confirmation.

coilysiren commented

2026-07-02 08:00:24 +00:00

Author

Owner

Correcting the diagnosis on this one. The observed local failure is not the port mapping - docker logs agent-proxy shows the container crash-looping on ModuleNotFoundError: No module named 'pydantic_settings', so it never binds any port at all. That root cause is a malformed dependency declaration in the agent-proxy image, tracked in coilyco-flight-deck/agent-proxy#22.

This port-mapping bug is still real, but it is latent - it only surfaces once the container actually boots. With agent_proxy_port: 8082 pushed into the container as PROXY_PORT, the app binds 0.0.0.0:8082 while compose.yaml maps host 8082 -> container 8080, so /v1/healthz would still fail even on a healthy image. Keep this fix (pin the container's internal port to 8080, vary only the published host port).

Ordering for a green end-to-end: agent-proxy#22 (deps) unblocks boot, this (#430) fixes the healthcheck path, and infrastructure#431 makes the role actually rebuild the image so the fixed upstream reaches the host. Same role/file as #431 - whichever lands second integrates the first.

Correcting the diagnosis on this one. The observed local failure is **not** the port mapping - `docker logs agent-proxy` shows the container crash-looping on `ModuleNotFoundError: No module named 'pydantic_settings'`, so it never binds any port at all. That root cause is a malformed dependency declaration in the agent-proxy image, tracked in coilyco-flight-deck/agent-proxy#22. This port-mapping bug is still real, but it is **latent** - it only surfaces once the container actually boots. With `agent_proxy_port: 8082` pushed into the container as `PROXY_PORT`, the app binds `0.0.0.0:8082` while `compose.yaml` maps host `8082` -> container `8080`, so `/v1/healthz` would still fail even on a healthy image. Keep this fix (pin the container's internal port to 8080, vary only the published host port). Ordering for a green end-to-end: agent-proxy#22 (deps) unblocks boot, this (#430) fixes the healthcheck path, and infrastructure#431 makes the role actually rebuild the image so the fixed upstream reaches the host. Same role/file as #431 - whichever lands second integrates the first.

coilysiren referenced this issue

2026-07-02 08:04:29 +00:00

agent-proxy role never rebuilds the image on converge (docker compose up --force-recreate, no --build/--pull) - upstream fixes never reach the host #431

coilysiren referenced this issue

2026-07-02 08:43:53 +00:00

agent-proxy role skips rebuild when a container is already running - new upstream builds never auto-pulled (trigger gate, plus no_log hides build errors) #434