coilyco-bridge/coily

Fork 0

Force package managers through local proxies with default-deny egress #64

New issue

Open

opened 2026-05-23 20:54:05 +00:00 by coilysiren · 0 comments

coilysiren commented

2026-05-23 20:54:05 +00:00

Owner

Originally filed by @coilysiren on 2026-04-29T08:54:26Z - https://github.com/coilysiren/coily/issues/35

🤖 Filed by Claude Code on Kai's behalf.

2026-04-30 update. Absorbed #33 (audit egress: log outbound network calls per wrapped command). One proxy serves both stories: enforce allowlists for the package-manager wrappers, observe-only for the other passthroughs (aws/gh/kubectl/docker/tailscale). Audit rows for both modes share the parent invocation's audit-row id.

Problem

Package managers (brew, npm, pip, cargo, gem, etc.) currently reach the internet directly. That's a wide unsupervised egress surface during agent runs. Any npm install, pip install, cargo install, or brew install can pull from arbitrary registries, mirrors, or postinstall scripts with no per-host gating. Lockdown's argv validation does not see what those subprocesses fetch.

The non-pkgmgr passthroughs (coily aws, coily gh, coily kubectl, coily docker, coily tailscale) have a softer version of the same gap: their argv is audited, the hosts they hit are not. A platform-engineer egress story is "which command went where," not "which command ran."

Proposal

One in-process Go HTTP CONNECT proxy that coily starts on 127.0.0.1:0 for the duration of a wrapped invocation. The child subprocess inherits HTTPS_PROXY / HTTP_PROXY pointing at the proxy. The proxy logs every CONNECT, joins the rows back to the parent invocation's audit-row id, and (per-binary) either enforces a default-deny allowlist or observes silently.

Two enforcement modes:

enforce for the 12 pkgmgr wrappers. Per-binary allowlist pinned to that manager's known-good upstreams. Denied CONNECTs return 403, the audit row marks decision=deny, and the underlying tool sees a connection failure.
observe for aws, gh, kubectl, docker, tailscale. No allowlist. Every CONNECT is forwarded and logged. This is the #33 story.

Allowlists are pinned in code, not user-configurable for v0.1. Defaults follow the original #35 sketch:

brew - formulae.brew.sh, ghcr.io, objects.githubusercontent.com, github.com, raw.githubusercontent.com
npm / pnpm / yarn / bun - registry.npmjs.org
pip / uv / pipx / poetry - pypi.org, files.pythonhosted.org
cargo - crates.io, static.crates.io, index.crates.io, github.com
gem / bundle - rubygems.org, index.rubygems.org

CONNECT-only. No TLS interception, no CA install on either Mac or Windows hosts. Hostnames come from the SNI / Host: line in the CONNECT verb.

Architecture

┌── coily <pkgmgr> args... ──────────────────────────────┐
│                                                        │
│  1. verb.Wrap starts                                   │
│  2. egress.Proxy.Start() -> 127.0.0.1:NNNNN            │
│  3. shell.Runner.Exec child with                       │
│       HTTPS_PROXY=http://127.0.0.1:NNNNN               │
│       HTTP_PROXY=http://127.0.0.1:NNNNN                │
│  4. child runs, hits proxy on every TLS dial           │
│       proxy: allow-or-deny per allowlist               │
│       proxy: collect EgressRow in memory               │
│  5. child exits                                        │
│  6. egress.Proxy.Stop() -> drains rows                 │
│  7. audit.Writer.Append parent record with             │
│       Egress: []EgressRow{...}                         │
└────────────────────────────────────────────────────────┘

Audit-row shape

Extend audit.Record with one optional field:

type Record struct {
    // ... existing fields ...
    Egress []EgressRow `json:"egress,omitempty"`
}

type EgressRow struct {
    Host       string `json:"host"`        // sni / host header from CONNECT
    Decision   string `json:"decision"`    // "allow" | "deny"
    BytesUp    int64  `json:"bytes_up"`
    BytesDown  int64  `json:"bytes_down"`
    DurationMS int64  `json:"duration_ms"`
}

One row per host contacted per parent invocation. Aggregation (one record per (parent_id, host) pair, summing bytes / durations) happens in the proxy before flushing to the parent record so a coily npm install run that opens 200 connections to registry.npmjs.org does not produce 200 audit rows.

Phases

Phase 1: tracer bullet (this issue, batch 1)

Stops with a pause so Kai can review before expanding to the other 11 pkgmgrs.

Scope:

pkg/egress/ new package:
- proxy.go - HTTP CONNECT proxy. New(allowlist []string, mode Mode) returns a *Proxy. Start(ctx) listens on 127.0.0.1:0 and returns the proxy URL. Stop() returns the collected []EgressRow. Hostname match: exact match against the allowlist plus suffix match for any allowlist entry beginning with *. (e.g. *.amazonaws.com). Tracer-bullet allowlist for brew is exact-match only; suffix matching can land in Phase 2.
- allowlist.go - per-binary allowlist as a Go map. var Allowlists = map[string][]string{ "brew": {...}, "npm": {...}, ... }. Phase 1 lands the brew entry; Phase 2 fills the rest.
- proxy_test.go - unit tests: allowed CONNECT forwards bytes, denied CONNECT returns 403 with audit row marked decision=deny, byte counters and duration populate.
pkg/audit/audit.go:
- Add Egress []EgressRow to Record. Add EgressRow struct.
- No change to Append/Wrap signatures. Caller populates Egress on the base record before writer.Wrap runs.
- Update audit_test.go to assert round-trip serialization includes egress rows when present and omits when absent.
pkg/shell/shell.go:
- Add Env []string field on Runner. When non-nil, Exec/Capture set cmd.Env = append(os.Environ(), r.Env...). When nil, default behavior (inherits os.Environ via cmd.Env == nil).
- Tests cover both nil and non-nil env paths.
pkg/ops/passthrough/passthrough.go:
- Add Mode type with ModeEnforce / ModeObserve constants.
- Add WithEgress(allowlist []string, mode Mode) option.
- When set, Command wraps the action: starts proxy, sets HTTPS_PROXY/HTTP_PROXY env on the shell.Runner for this invocation only (use a per-call shadow Runner so concurrent commands stay independent), runs the underlying action, stops proxy, attaches collected rows to the audit base record.
- The audit-base mutation is the awkward part. Two options: (a) plumb the egress rows out via a return value and have passthrough.Command build its own verb.Spec that decorates the base record post-action, or (b) add an OnComplete hook to verb.Spec that runs after Action returns and gets a chance to mutate the record. Implement (b); it's the cleaner seam and unlocks future per-verb side-channel data without re-plumbing.
pkg/verb/verb.go:
- Add OnComplete func(*audit.Record) to Spec. Called inside writer.Wrap after fn() returns, with a pointer to the record being finalized. Update verb_test.go to cover the hook.
cmd/coily/ops_pkgmgrs.go:
- Wire passthrough.WithEgress(egress.Allowlists["brew"], passthrough.ModeEnforce) for brew only in Phase 1.
- Other 11 pkgmgrs stay unchanged until Phase 2.
docs/features/25-egress-proxy.md new feature doc:
- Explain the proxy, the modes, the allowlist source of truth, the audit-row join, and the "no CA install" property. Cite #35 and #33 in the header.
End-to-end smoke test (manual, not committed as a test): on Mac, run coily brew search wget and coily brew install jq (or another tiny formula); verify the audit row contains egress rows for the expected hosts; run coily brew install some-formula-pulling-from-an-unallowed-host to confirm a deny path actually fails the install.

Pause for Kai's confirmation before Phase 2.

Phase 2: full sweep (after confirmation)

Fill pkg/egress/allowlist.go with the remaining 11 pkgmgr entries.
Wire passthrough.WithEgress(...) on the remaining 11 pkgmgr commands in cmd/coily/ops_pkgmgrs.go.
Wire passthrough.WithEgress(nil, passthrough.ModeObserve) on aws, gh, kubectl, docker, tailscale. Allowlist is nil in observe mode; every host is forwarded and logged.
Add suffix matching (*.foo.com) to egress.Proxy if any pkgmgr's real-world host pattern needs it. Likely yes for cargo (some crates fetch from *.crates.io).
Extend cmd/coily/ops_audit.go and the coily audit show / coily audit tail rendering paths to surface egress rows in human-readable output. (If coily audit show doesn't exist yet per the unresolved.md "What I would build next" #1, add it as part of this phase.)
Backfill docs/features/25-egress-proxy.md with the full surface, the suffix-match semantics, the observe-mode story, and a sample audit row.

Acceptance

Phase 1:

pkg/egress/ lands with tests passing.
coily brew search wget runs end-to-end and produces an audit row with egress: [{host: formulae.brew.sh, decision: allow, ...}, ...].
coily brew install <tiny-formula> runs end-to-end and audits the expected hosts.
An attempt to fetch from an unallowed host (synthetic test, not a real install) lands an egress: [{host: ..., decision: deny}] row and the underlying CONNECT returns 403.
No CA install required.
Proxy adds <50ms to a coily brew --version invocation.

Phase 2:

All 12 pkgmgrs wired in enforce mode.
All 5 non-pkgmgr passthroughs wired in observe mode.
coily aws s3 ls records one egress row per region endpoint contacted.
coily gh issue list records one egress row for api.github.com.
coily audit show <id> renders argv + egress together (or the rendering path is filed as its own follow-up issue if audit show doesn't yet exist).
docs/features/25-egress-proxy.md covers the full surface.
#33 closed as completed by this issue.

Out of scope

TLS MITM / request-path audit. Would require per-CLI CA plumbing (AWS_CA_BUNDLE, kubeconfig CAs, system trust for gh). Deliberate non-goal.
Per-process firewall enforcement (pf, socketfilterfw). Layered defense, not this layer.
Postinstall script sandboxing. Different problem, sandbox layer.
Generic outbound egress firewall for the whole machine. Different layer.
User-facing allowlist override config. Pinned in code for v0.1; can grow into ~/.coily/config.yaml later if needed.
Tailscale app-connector integration. Considered and rejected previously: per-node identity, not per-process; paid tier; flow logs don't join to argv.

Notes

The "lockdown wires the env vars" framing in the original #35 sketch is replaced by "the passthrough wrapper wires the env vars." Lockdown already denies the raw pkgmgr binaries (Bash(brew:*), Bash(npm:*), etc.), so the only path to invoke them is through coily <pkgmgr>, and the wrapper's job is to set HTTPS_PROXY before exec. No lockdown rule changes are needed for this issue.

_Originally filed by @coilysiren on 2026-04-29T08:54:26Z - [https://github.com/coilysiren/coily/issues/35](https://github.com/coilysiren/coily/issues/35)_ > 🤖 Filed by Claude Code on Kai's behalf. > **2026-04-30 update.** Absorbed [#33](https://github.com/coilysiren/coily/issues/33) (audit egress: log outbound network calls per wrapped command). One proxy serves both stories: enforce allowlists for the package-manager wrappers, observe-only for the other passthroughs (aws/gh/kubectl/docker/tailscale). Audit rows for both modes share the parent invocation's audit-row id. ## Problem Package managers (brew, npm, pip, cargo, gem, etc.) currently reach the internet directly. That's a wide unsupervised egress surface during agent runs. Any `npm install`, `pip install`, `cargo install`, or `brew install` can pull from arbitrary registries, mirrors, or postinstall scripts with no per-host gating. Lockdown's argv validation does not see what those subprocesses fetch. The non-pkgmgr passthroughs (`coily aws`, `coily gh`, `coily kubectl`, `coily docker`, `coily tailscale`) have a softer version of the same gap: their argv is audited, the hosts they hit are not. A platform-engineer egress story is "which command went where," not "which command ran." ## Proposal One in-process Go HTTP CONNECT proxy that coily starts on `127.0.0.1:0` for the duration of a wrapped invocation. The child subprocess inherits `HTTPS_PROXY` / `HTTP_PROXY` pointing at the proxy. The proxy logs every CONNECT, joins the rows back to the parent invocation's audit-row id, and (per-binary) either enforces a default-deny allowlist or observes silently. Two enforcement modes: - **enforce** for the 12 pkgmgr wrappers. Per-binary allowlist pinned to that manager's known-good upstreams. Denied CONNECTs return 403, the audit row marks `decision=deny`, and the underlying tool sees a connection failure. - **observe** for `aws`, `gh`, `kubectl`, `docker`, `tailscale`. No allowlist. Every CONNECT is forwarded and logged. This is the [#33](https://github.com/coilysiren/coily/issues/33) story. Allowlists are pinned in code, not user-configurable for v0.1. Defaults follow the original [#35](https://github.com/coilysiren/coily/issues/35) sketch: - **brew** - `formulae.brew.sh`, `ghcr.io`, `objects.githubusercontent.com`, `github.com`, `raw.githubusercontent.com` - **npm / pnpm / yarn / bun** - `registry.npmjs.org` - **pip / uv / pipx / poetry** - `pypi.org`, `files.pythonhosted.org` - **cargo** - `crates.io`, `static.crates.io`, `index.crates.io`, `github.com` - **gem / bundle** - `rubygems.org`, `index.rubygems.org` CONNECT-only. No TLS interception, no CA install on either Mac or Windows hosts. Hostnames come from the SNI / `Host:` line in the CONNECT verb. ## Architecture ``` ┌── coily <pkgmgr> args... ──────────────────────────────┐ │ │ │ 1. verb.Wrap starts │ │ 2. egress.Proxy.Start() -> 127.0.0.1:NNNNN │ │ 3. shell.Runner.Exec child with │ │ HTTPS_PROXY=http://127.0.0.1:NNNNN │ │ HTTP_PROXY=http://127.0.0.1:NNNNN │ │ 4. child runs, hits proxy on every TLS dial │ │ proxy: allow-or-deny per allowlist │ │ proxy: collect EgressRow in memory │ │ 5. child exits │ │ 6. egress.Proxy.Stop() -> drains rows │ │ 7. audit.Writer.Append parent record with │ │ Egress: []EgressRow{...} │ └────────────────────────────────────────────────────────┘ ``` ## Audit-row shape Extend `audit.Record` with one optional field: ```go type Record struct { // ... existing fields ... Egress []EgressRow `json:"egress,omitempty"` } type EgressRow struct { Host string `json:"host"` // sni / host header from CONNECT Decision string `json:"decision"` // "allow" | "deny" BytesUp int64 `json:"bytes_up"` BytesDown int64 `json:"bytes_down"` DurationMS int64 `json:"duration_ms"` } ``` One row per host contacted per parent invocation. Aggregation (one record per `(parent_id, host)` pair, summing bytes / durations) happens in the proxy before flushing to the parent record so a `coily npm install` run that opens 200 connections to `registry.npmjs.org` does not produce 200 audit rows. ## Phases ### Phase 1: tracer bullet (this issue, batch 1) **Stops with a pause** so Kai can review before expanding to the other 11 pkgmgrs. Scope: 1. **`pkg/egress/`** new package: - `proxy.go` - HTTP CONNECT proxy. `New(allowlist []string, mode Mode)` returns a `*Proxy`. `Start(ctx)` listens on `127.0.0.1:0` and returns the proxy URL. `Stop()` returns the collected `[]EgressRow`. Hostname match: exact match against the allowlist plus suffix match for any allowlist entry beginning with `*.` (e.g. `*.amazonaws.com`). Tracer-bullet allowlist for brew is exact-match only; suffix matching can land in Phase 2. - `allowlist.go` - per-binary allowlist as a Go map. `var Allowlists = map[string][]string{ "brew": {...}, "npm": {...}, ... }`. Phase 1 lands the brew entry; Phase 2 fills the rest. - `proxy_test.go` - unit tests: allowed CONNECT forwards bytes, denied CONNECT returns 403 with audit row marked `decision=deny`, byte counters and duration populate. 2. **`pkg/audit/audit.go`**: - Add `Egress []EgressRow` to `Record`. Add `EgressRow` struct. - No change to `Append`/`Wrap` signatures. Caller populates `Egress` on the base record before `writer.Wrap` runs. - Update `audit_test.go` to assert round-trip serialization includes egress rows when present and omits when absent. 3. **`pkg/shell/shell.go`**: - Add `Env []string` field on `Runner`. When non-nil, `Exec`/`Capture` set `cmd.Env = append(os.Environ(), r.Env...)`. When nil, default behavior (inherits os.Environ via `cmd.Env == nil`). - Tests cover both nil and non-nil env paths. 4. **`pkg/ops/passthrough/passthrough.go`**: - Add `Mode` type with `ModeEnforce` / `ModeObserve` constants. - Add `WithEgress(allowlist []string, mode Mode)` option. - When set, `Command` wraps the action: starts proxy, sets `HTTPS_PROXY`/`HTTP_PROXY` env on the shell.Runner for this invocation only (use a per-call shadow Runner so concurrent commands stay independent), runs the underlying action, stops proxy, attaches collected rows to the audit base record. - The audit-base mutation is the awkward part. Two options: (a) plumb the egress rows out via a return value and have `passthrough.Command` build its own `verb.Spec` that decorates the base record post-action, or (b) add an `OnComplete` hook to `verb.Spec` that runs after `Action` returns and gets a chance to mutate the record. Implement (b); it's the cleaner seam and unlocks future per-verb side-channel data without re-plumbing. 5. **`pkg/verb/verb.go`**: - Add `OnComplete func(*audit.Record)` to `Spec`. Called inside `writer.Wrap` after `fn()` returns, with a pointer to the record being finalized. Update `verb_test.go` to cover the hook. 6. **`cmd/coily/ops_pkgmgrs.go`**: - Wire `passthrough.WithEgress(egress.Allowlists["brew"], passthrough.ModeEnforce)` for `brew` only in Phase 1. - Other 11 pkgmgrs stay unchanged until Phase 2. 7. **`docs/features/25-egress-proxy.md`** new feature doc: - Explain the proxy, the modes, the allowlist source of truth, the audit-row join, and the "no CA install" property. Cite [#35](https://github.com/coilysiren/coily/issues/35) and [#33](https://github.com/coilysiren/coily/issues/33) in the header. 8. **End-to-end smoke test** (manual, not committed as a test): on Mac, run `coily brew search wget` and `coily brew install jq` (or another tiny formula); verify the audit row contains egress rows for the expected hosts; run `coily brew install some-formula-pulling-from-an-unallowed-host` to confirm a deny path actually fails the install. **Pause for Kai's confirmation before Phase 2.** ### Phase 2: full sweep (after confirmation) 1. Fill `pkg/egress/allowlist.go` with the remaining 11 pkgmgr entries. 2. Wire `passthrough.WithEgress(...)` on the remaining 11 pkgmgr commands in `cmd/coily/ops_pkgmgrs.go`. 3. Wire `passthrough.WithEgress(nil, passthrough.ModeObserve)` on `aws`, `gh`, `kubectl`, `docker`, `tailscale`. Allowlist is nil in observe mode; every host is forwarded and logged. 4. Add suffix matching (`*.foo.com`) to `egress.Proxy` if any pkgmgr's real-world host pattern needs it. Likely yes for cargo (some crates fetch from `*.crates.io`). 5. Extend `cmd/coily/ops_audit.go` and the `coily audit show` / `coily audit tail` rendering paths to surface egress rows in human-readable output. (If `coily audit show` doesn't exist yet per the unresolved.md "What I would build next" #1, add it as part of this phase.) 6. Backfill `docs/features/25-egress-proxy.md` with the full surface, the suffix-match semantics, the observe-mode story, and a sample audit row. ## Acceptance Phase 1: - [ ] `pkg/egress/` lands with tests passing. - [ ] `coily brew search wget` runs end-to-end and produces an audit row with `egress: [{host: formulae.brew.sh, decision: allow, ...}, ...]`. - [ ] `coily brew install <tiny-formula>` runs end-to-end and audits the expected hosts. - [ ] An attempt to fetch from an unallowed host (synthetic test, not a real install) lands an `egress: [{host: ..., decision: deny}]` row and the underlying CONNECT returns 403. - [ ] No CA install required. - [ ] Proxy adds <50ms to a `coily brew --version` invocation. Phase 2: - [ ] All 12 pkgmgrs wired in enforce mode. - [ ] All 5 non-pkgmgr passthroughs wired in observe mode. - [ ] `coily aws s3 ls` records one egress row per region endpoint contacted. - [ ] `coily gh issue list` records one egress row for `api.github.com`. - [ ] `coily audit show <id>` renders argv + egress together (or the rendering path is filed as its own follow-up issue if `audit show` doesn't yet exist). - [ ] `docs/features/25-egress-proxy.md` covers the full surface. - [ ] [#33](https://github.com/coilysiren/coily/issues/33) closed as completed by this issue. ## Out of scope - TLS MITM / request-path audit. Would require per-CLI CA plumbing (`AWS_CA_BUNDLE`, kubeconfig CAs, system trust for `gh`). Deliberate non-goal. - Per-process firewall enforcement (pf, socketfilterfw). Layered defense, not this layer. - Postinstall script sandboxing. Different problem, sandbox layer. - Generic outbound egress firewall for the whole machine. Different layer. - User-facing allowlist override config. Pinned in code for v0.1; can grow into `~/.coily/config.yaml` later if needed. - Tailscale app-connector integration. Considered and rejected previously: per-node identity, not per-process; paid tier; flow logs don't join to argv. ## Notes The "lockdown wires the env vars" framing in the original [#35](https://github.com/coilysiren/coily/issues/35) sketch is replaced by "the passthrough wrapper wires the env vars." Lockdown already denies the raw pkgmgr binaries (`Bash(brew:*)`, `Bash(npm:*)`, etc.), so the only path to invoke them is through `coily <pkgmgr>`, and the wrapper's job is to set `HTTPS_PROXY` before exec. No lockdown rule changes are needed for this issue.

coilysiren added the

label

2026-05-31 01:52:43 +00:00

coilysiren added

and removed

labels

2026-05-31 06:59:47 +00:00

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

coilyco-bridge/coily#64

No description provided.

Rows
Columns