logdy: live-discover new agent-run dirs without a container restart

coilysiren commented

2026-07-01 06:59:30 +00:00

Owner

Split off from #420 (the Logdy filesystem log-viewer role).

The logdy role runs logdy follow --full-read over the drained
~/.ward/agent-logs/ tree. follow takes a fixed file list resolved once
at container start (Logdy has no glob / directory-watch), so runs that drain
after the container comes up are not visible until the next re-converge or
docker compose restart logdy.

For the "always-up, always-current" ideal we want live discovery of new run
dirs without a restart. Options to weigh:

a thin watcher sidecar (inotify / fswatch) that docker compose restarts
logdy when a new <container>/ dir appears (simple, but drops the UI buffer),
pivot to Logdy stdin mode fed by tail -F with retry over a shell that
re-globs, or a find-loop feeder,
upstream feature request to Logdy for a directory-follow / glob mode.

Until then the documented workaround (re-converge to pick up freshly closed
runs) is in docs/FEATURES.md. Low priority - closed runs that existed at
start are already browsable, which was the point of #420.

Split off from #420 (the Logdy filesystem log-viewer role). The `logdy` role runs `logdy follow --full-read` over the drained `~/.ward/agent-logs/` tree. `follow` takes a **fixed file list** resolved once at container start (Logdy has no glob / directory-watch), so runs that drain **after** the container comes up are not visible until the next re-converge or `docker compose restart logdy`. For the "always-up, always-current" ideal we want live discovery of new run dirs without a restart. Options to weigh: - a thin watcher sidecar (inotify / `fswatch`) that `docker compose restart`s logdy when a new `<container>/` dir appears (simple, but drops the UI buffer), - pivot to Logdy `stdin` mode fed by `tail -F` with retry over a shell that re-globs, or a `find`-loop feeder, - upstream feature request to Logdy for a directory-follow / glob mode. Until then the documented workaround (re-converge to pick up freshly closed runs) is in `docs/FEATURES.md`. Low priority - closed runs that existed at start are already browsable, which was the point of #420.

coilysiren referenced this issue from a commit

2026-07-01 07:02:05 +00:00

feat(ansible): add logdy filesystem log-viewer role

coilysiren referenced this issue

2026-07-01 07:02:27 +00:00

ansible: add a Logdy always-up log-viewer role (browse ~/.ward/agent-logs, closed runs included) #420

coilyco-ops added the

consult

P4

labels

2026-07-01 08:07:57 +00:00

coilysiren referenced this issue

2026-07-01 08:18:04 +00:00

Point Dozzle (or document grep/jq path) at ~/.ward/agent-logs/ for ward agent-run drains (ward#363) #413

coilysiren commented

2026-07-01 08:24:00 +00:00

Author

Owner

Decision: Option A (watcher-driven logdy restart)

Dug into the three options against logdy v0.17.1. Confirmed upstream still has no runtime directory-follow or glob-rescan: follow resolves a fixed file list once at start, and the logdy follow *.log shell-glob is expanded by the shell (so it hits the same fixed-at-start wall we already have). So Option C's upstream feature does not exist to lean on today.

Between the two local options, the deciding factor is that Option B (stdin feeder) regresses the UX: logdy stdin mode carries no per-file origin metadata, so we would have to prefix each line with its path, which mangles the transcript.jsonl JSON and breaks the in-browser TypeScript parsers that make the per-run view readable. Option A keeps native follow --full-read, so origin tagging and the JSON parsers stay intact. The only cost is a brief UI reset per restart, and --full-read rebuilds the line buffer on restart so no history is lost.

Recommended shape (no docker socket, no host launchd)

Keep it inside the one logdy container rather than adding a socket-mounted sidecar (we deliberately avoid docker-socket exposure) or a host-level launchd job. Wrap the logdy command in a self-contained supervisor loop in compose.yaml:

compute a signature of the current matched file set (sorted list, or count + newest mtime) via the same find we run today,
start logdy ... follow --full-read $files in the background,
poll /logs on an interval (10-15s is fine for a browse-closed-runs tool), and when the signature changes, kill logdy and re-loop so it re-globs and re-execs,
debounce so a burst of draining runs triggers one restart, not a thrash.

Poll (not inotify) so we need no extra packages in the upstream-built image. The empty-tree stdin fallback stays as-is. This makes "container restart" an in-container logdy re-exec, delivering Option A's behavior with none of the socket or host-agent cost.

Dispatching a warded engineer run to implement this and update docs/FEATURES.md + the compose header comment (the current text points here as the open follow-up).

## Decision: Option A (watcher-driven logdy restart) Dug into the three options against logdy v0.17.1. Confirmed upstream still has **no** runtime directory-follow or glob-rescan: `follow` resolves a fixed file list once at start, and the `logdy follow *.log` shell-glob is expanded by the shell (so it hits the same fixed-at-start wall we already have). So Option C's upstream feature does not exist to lean on today. Between the two local options, the deciding factor is that **Option B (stdin feeder) regresses the UX**: logdy `stdin` mode carries no per-file origin metadata, so we would have to prefix each line with its path, which mangles the `transcript.jsonl` JSON and breaks the in-browser TypeScript parsers that make the per-run view readable. Option A keeps native `follow --full-read`, so origin tagging and the JSON parsers stay intact. The only cost is a brief UI reset per restart, and `--full-read` rebuilds the line buffer on restart so no history is lost. ### Recommended shape (no docker socket, no host launchd) Keep it inside the one logdy container rather than adding a socket-mounted sidecar (we deliberately avoid docker-socket exposure) or a host-level launchd job. Wrap the logdy command in a self-contained supervisor loop in `compose.yaml`: - compute a signature of the current matched file set (sorted list, or count + newest mtime) via the same `find` we run today, - start `logdy ... follow --full-read $files` in the background, - poll `/logs` on an interval (10-15s is fine for a browse-closed-runs tool), and when the signature changes, kill logdy and re-loop so it re-globs and re-execs, - debounce so a burst of draining runs triggers one restart, not a thrash. Poll (not inotify) so we need no extra packages in the upstream-built image. The empty-tree stdin fallback stays as-is. This makes "container restart" an in-container logdy re-exec, delivering Option A's behavior with none of the socket or host-agent cost. Dispatching a `warded engineer` run to implement this and update `docs/FEATURES.md` + the compose header comment (the current text points here as the open follow-up).

coilyco-ops commented

2026-07-01 08:24:19 +00:00

Member

🔒 Reserved by ward agent --driver claude — container engineer-claude-infrastructure-421 on host kais-macbook-pro-2.local is carrying this issue (reserved 2026-07-01T08:24:16Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); --force overrides.

— Claude (she/her), via ward agent

🔒 Reserved by `ward agent --driver claude` — container `engineer-claude-infrastructure-421` on host `kais-macbook-pro-2.local` is carrying this issue (reserved 2026-07-01T08:24:16Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); `--force` overrides.  — Claude (she/her), via `ward agent`

coilysiren referenced this issue from a commit

2026-07-01 08:31:32 +00:00

feat(logdy): live-discover new agent-run dirs without a container restart

coilysiren closed this issue

2026-07-01 08:31:32 +00:00

coilysiren referenced this issue from a commit

2026-07-01 08:31:32 +00:00

Merge issue-421: logdy live-discover new agent-run dirs (closes #421)

coilysiren commented

2026-07-01 08:33:36 +00:00

Author

Owner

WARD-OUTCOME: done - logdy now live-discovers new run dirs via an in-container poll/restart supervisor, no re-converge needed

The design decision (Option A, an in-container supervisor loop) was already settled in the thread, so this was mostly a shell-craft exercise rather than a research one. The genuinely fiddly part was the empty-tree fallback: a naive sleep infinity | logdy & pipeline leaks an orphaned feeder process every time the tree goes from empty to populated. Feeding logdy's stdin from a FIFO held open on fd 3 sidesteps that cleanly, and stubbing out logdy let me actually prove zero orphans across both the populated-restart and empty-then-first-run paths.

One thing that fought back but was not my code: the pylint pre-commit hook kept failing on Permission denied writing /opt/uv/python in this container. It reproduces on an unmodified tree, so it is environmental, and pointing UV_PYTHON_INSTALL_DIR at a writable dir clears it. Worth noting for anyone else committing from a ward box.

Confidence is high on the supervisor logic since I tested the state machine directly, and CI plus pre-commit are green. The one thing I could not verify here is real logdy behavior: no compose plugin or image build in this container, so I am trusting that logdy tolerates a FIFO on stdin in follow mode and releases port 8080 promptly on SIGTERM (the kill-wait-start sequencing is there to cover the latter). If either assumption is wrong it will show up the first time a run drains on a live mac.

Rough edge, as flagged in the issue: each restart drops the in-UI scroll buffer. That is inherent to Option A. If it turns out to grate in daily use, the real fix is still the upstream directory-follow feature request (option C) rather than more supervisor cleverness.

WARD-OUTCOME: done - logdy now live-discovers new run dirs via an in-container poll/restart supervisor, no re-converge needed The design decision (Option A, an in-container supervisor loop) was already settled in the thread, so this was mostly a shell-craft exercise rather than a research one. The genuinely fiddly part was the empty-tree fallback: a naive `sleep infinity | logdy &` pipeline leaks an orphaned feeder process every time the tree goes from empty to populated. Feeding logdy's stdin from a FIFO held open on fd 3 sidesteps that cleanly, and stubbing out logdy let me actually prove zero orphans across both the populated-restart and empty-then-first-run paths. One thing that fought back but was not my code: the pylint pre-commit hook kept failing on `Permission denied` writing `/opt/uv/python` in this container. It reproduces on an unmodified tree, so it is environmental, and pointing `UV_PYTHON_INSTALL_DIR` at a writable dir clears it. Worth noting for anyone else committing from a ward box. Confidence is high on the supervisor logic since I tested the state machine directly, and CI plus pre-commit are green. The one thing I could not verify here is real logdy behavior: no compose plugin or image build in this container, so I am trusting that logdy tolerates a FIFO on stdin in follow mode and releases port 8080 promptly on SIGTERM (the kill-wait-start sequencing is there to cover the latter). If either assumption is wrong it will show up the first time a run drains on a live mac. Rough edge, as flagged in the issue: each restart drops the in-UI scroll buffer. That is inherent to Option A. If it turns out to grate in daily use, the real fix is still the upstream directory-follow feature request (option C) rather than more supervisor cleverness.

coilysiren referenced this issue

2026-07-05 03:34:12 +00:00

Decide whether to retire the logdy role now that ward retired the disk+logdy workflow #473

Rows
Columns

logdy: live-discover new agent-run dirs without a container restart #421

Decision: Option A (watcher-driven logdy restart)

Recommended shape (no docker socket, no host launchd)