coilyco-flight-deck/infrastructure
2
0
Fork 0
Code Issues 118 Pull requests Projects Releases Packages Wiki Activity Actions 2

Rebuild Forgejo PATs under the coilyco-ops bot (after the token wipe) #384

New issue
Closed
opened 2026-06-19 11:03:22 +00:00 by coilyco-ops · 3 comments
coilyco-ops commented 2026-06-19 11:03:22 +00:00
Member
Copy link

Context

All coilysiren personal Forgejo PATs were deleted (after an all-scopes admin token leaked into a build log via set -eux - see agentic-os#223 fix e329ba8). Nothing is insecure now, just broken: anything that read those tokens is down. Rebuilding bot-centric (coilyco-ops) instead of personal.

Current state (verified)

  • ALIVE: /forgejo/coilyco-ops/api-token (bot PAT - ward ops forgejo still works) and /forgejo/coilyco-ops/password. The bot has push on org repos (permissions: {admin:false, pull:true, push:true}), so it is a viable replacement for coilysiren.
  • DEAD (deleted coilysiren PATs, SSM values now invalid): /forgejo/api-token (was the admin/all token), /forgejo/ci-release-token, /forgejo/flux-deploy-token, /forgejo/registry-read-token.
  • NOT PATs (Forgejo server-config secrets, untouched): internal-token, metrics-token, secret-key, lfs-jwt-secret, db-password, oauth-github/*.

Target token set (least privilege, all under coilyco-ops)

SSM path token name scope
/forgejo/api-token ward-container-push write:repository
/forgejo/ci-release-token ci-release write:repository
/forgejo/flux-deploy-token flux-deploy read:repository
/forgejo/registry-read-token registry-read read:package

No more all/admin token in the hot path. Mint a separate real admin token by hand only if/when needed.

Step 1 - mint the tokens (run on a host with AWS + the bot password in SSM)

Saves to the same SSM paths so consumers keep reading them. Token values never hit stdout/argv (basic auth via curlrc, .sha1 -> 0600 cli-input-json -> ssm put-parameter).

#!/usr/bin/env bash
# Mint per-consumer least-privilege coilyco-ops tokens -> SSM.
set -euo pipefail
BASE="https://forgejo.coilysiren.me"; BOT="coilyco-ops"
work="$(mktemp -d)"; chmod 700 "$work"; trap 'rm -rf "$work"' EXIT
aws ssm get-parameter --name /forgejo/coilyco-ops/password --with-decryption \
  --query Parameter.Value --output text > "$work/pw"
printf 'user = "%s:%s"\n' "$BOT" "$(cat "$work/pw")" > "$work/rc"; chmod 600 "$work/pw" "$work/rc"
mint() { # name  scopes-json  ssmpath
  echo "== minting '$1' ($2) -> $3 =="
  local resp; resp="$(curl -fsS -K "$work/rc" -X POST -H 'Content-Type: application/json' \
    -d "{\"name\":\"$1\",\"scopes\":$2}" "$BASE/api/v1/users/$BOT/tokens" 2>&1)" \
    || { echo "  MINT FAILED: $(printf '%s' "$resp" | head -1)"; return 1; }
  printf '%s' "$resp" | jq --arg n "$3" '{Name:$n,Value:.sha1,Type:"SecureString",Overwrite:true}' > "$work/put.json"
  chmod 600 "$work/put.json"; aws ssm put-parameter --cli-input-json "file://$work/put.json" >/dev/null
  echo "  OK: $1 -> $3 (last8 $(printf '%s' "$resp" | jq -r .token_last_eight))"
}
mint ward-container-push '["write:repository"]' /forgejo/api-token
mint ci-release          '["write:repository"]' /forgejo/ci-release-token
mint flux-deploy         '["read:repository"]'  /forgejo/flux-deploy-token
mint registry-read       '["read:package"]'     /forgejo/registry-read-token
echo "== done: 4 bot tokens minted + written to SSM =="

Step 2 - repoint the ward git cred-helper (code; coilyco-flight-deck/ward)

The git-over-HTTPS push user is hardcoded coilysiren in two spots; the username must match the new token's owner (coilyco-ops):

  • cmd/ward/containerassets/entrypoint.sh - printf 'https://%s:%s@%s\n' coilysiren ... -> coilyco-ops
  • cmd/ward/container_bootstrap.go - fmt.Sprintf("https://%s:%s@%s\n", "coilysiren", ...) -> coilyco-ops

(Also worth flipping the registry login user in agentic-os .forgejo/workflows/release.yml from coilysiren to coilyco-ops - already staged on the issue-223 branch.)

Step 3 - CI Actions secrets (admin-only; needs a human)

The bot is admin:false, so it cannot set Actions secrets via API. For each repo's release/registry workflow, set the Actions secret to the matching new bot token value:

  • REGISTRY_TOKEN (registry push) - mint a write:package bot token for this (not in the SSM set above; CI-only).
  • CI_RELEASE_TOKEN / FORGEJO_PUSH_TOKEN - the ci-release token value.
  • Audit the full secret list found across repos: CI_RELEASE_TOKEN, FORGEJO_PUSH_TOKEN, REGISTRY_TOKEN, GH_PUSH_TOKEN, DATASTORE_TOKEN, DOCKERHUB_* - confirm which are Forgejo PATs vs other.

Step 4 - validate

  • ward push: a real ward agent claude headless <issue> run lands a commit as coilyco-ops.
  • CI: a release runs green (release + publish-image) once secrets are repointed.
  • flux: GitOps reconciles against the deploy repo with the read token.

Related

  • infrastructure#383 (auto-rotate bot creds + source CI token from the rotating value) - this is the manual rebuild that #383 should eventually automate.
  • infrastructure#382 (bot org membership/repo-create grant) - bot now has push; confirm package-write + any admin needs.
  • agentic-os#223 (bake ward via bot creds) and the leak fix e329ba8.
## Context All **coilysiren** personal Forgejo PATs were deleted (after an `all`-scopes `admin` token leaked into a build log via `set -eux` - see agentic-os#223 fix `e329ba8`). Nothing is insecure now, just **broken**: anything that read those tokens is down. Rebuilding bot-centric (`coilyco-ops`) instead of personal. ## Current state (verified) - **ALIVE**: `/forgejo/coilyco-ops/api-token` (bot PAT - `ward ops forgejo` still works) and `/forgejo/coilyco-ops/password`. The bot has **push** on org repos (`permissions: {admin:false, pull:true, push:true}`), so it is a viable replacement for coilysiren. - **DEAD** (deleted coilysiren PATs, SSM values now invalid): `/forgejo/api-token` (was the `admin`/all token), `/forgejo/ci-release-token`, `/forgejo/flux-deploy-token`, `/forgejo/registry-read-token`. - **NOT PATs** (Forgejo server-config secrets, untouched): `internal-token`, `metrics-token`, `secret-key`, `lfs-jwt-secret`, `db-password`, `oauth-github/*`. ## Target token set (least privilege, all under `coilyco-ops`) | SSM path | token name | scope | |---|---|---| | `/forgejo/api-token` | `ward-container-push` | `write:repository` | | `/forgejo/ci-release-token` | `ci-release` | `write:repository` | | `/forgejo/flux-deploy-token` | `flux-deploy` | `read:repository` | | `/forgejo/registry-read-token` | `registry-read` | `read:package` | No more `all`/admin token in the hot path. Mint a separate real admin token by hand only if/when needed. ## Step 1 - mint the tokens (run on a host with AWS + the bot password in SSM) Saves to the same SSM paths so consumers keep reading them. Token values never hit stdout/argv (basic auth via curlrc, `.sha1` -> 0600 `cli-input-json` -> `ssm put-parameter`). ```bash #!/usr/bin/env bash # Mint per-consumer least-privilege coilyco-ops tokens -> SSM. set -euo pipefail BASE="https://forgejo.coilysiren.me"; BOT="coilyco-ops" work="$(mktemp -d)"; chmod 700 "$work"; trap 'rm -rf "$work"' EXIT aws ssm get-parameter --name /forgejo/coilyco-ops/password --with-decryption \ --query Parameter.Value --output text > "$work/pw" printf 'user = "%s:%s"\n' "$BOT" "$(cat "$work/pw")" > "$work/rc"; chmod 600 "$work/pw" "$work/rc" mint() { # name scopes-json ssmpath echo "== minting '$1' ($2) -> $3 ==" local resp; resp="$(curl -fsS -K "$work/rc" -X POST -H 'Content-Type: application/json' \ -d "{\"name\":\"$1\",\"scopes\":$2}" "$BASE/api/v1/users/$BOT/tokens" 2>&1)" \ || { echo " MINT FAILED: $(printf '%s' "$resp" | head -1)"; return 1; } printf '%s' "$resp" | jq --arg n "$3" '{Name:$n,Value:.sha1,Type:"SecureString",Overwrite:true}' > "$work/put.json" chmod 600 "$work/put.json"; aws ssm put-parameter --cli-input-json "file://$work/put.json" >/dev/null echo " OK: $1 -> $3 (last8 $(printf '%s' "$resp" | jq -r .token_last_eight))" } mint ward-container-push '["write:repository"]' /forgejo/api-token mint ci-release '["write:repository"]' /forgejo/ci-release-token mint flux-deploy '["read:repository"]' /forgejo/flux-deploy-token mint registry-read '["read:package"]' /forgejo/registry-read-token echo "== done: 4 bot tokens minted + written to SSM ==" ``` ## Step 2 - repoint the ward git cred-helper (code; coilyco-flight-deck/ward) The git-over-HTTPS push user is hardcoded `coilysiren` in two spots; the username must match the new token's owner (`coilyco-ops`): - `cmd/ward/containerassets/entrypoint.sh` - `printf 'https://%s:%s@%s\n' coilysiren ...` -> `coilyco-ops` - `cmd/ward/container_bootstrap.go` - `fmt.Sprintf("https://%s:%s@%s\n", "coilysiren", ...)` -> `coilyco-ops` (Also worth flipping the registry login user in agentic-os `.forgejo/workflows/release.yml` from `coilysiren` to `coilyco-ops` - already staged on the `issue-223` branch.) ## Step 3 - CI Actions secrets (admin-only; needs a human) The bot is `admin:false`, so it cannot set Actions secrets via API. For each repo's release/registry workflow, set the Actions secret to the matching new bot token value: - `REGISTRY_TOKEN` (registry push) - mint a `write:package` bot token for this (not in the SSM set above; CI-only). - `CI_RELEASE_TOKEN` / `FORGEJO_PUSH_TOKEN` - the `ci-release` token value. - Audit the full secret list found across repos: `CI_RELEASE_TOKEN, FORGEJO_PUSH_TOKEN, REGISTRY_TOKEN, GH_PUSH_TOKEN, DATASTORE_TOKEN, DOCKERHUB_*` - confirm which are Forgejo PATs vs other. ## Step 4 - validate - ward push: a real `ward agent claude headless <issue>` run lands a commit as `coilyco-ops`. - CI: a release runs green (release + publish-image) once secrets are repointed. - flux: GitOps reconciles against the deploy repo with the read token. ## Related - infrastructure#383 (auto-rotate bot creds + source CI token from the rotating value) - this is the manual rebuild that #383 should eventually automate. - infrastructure#382 (bot org membership/repo-create grant) - bot now has push; confirm package-write + any admin needs. - agentic-os#223 (bake ward via bot creds) and the leak fix `e329ba8`.
coilyco-ops commented 2026-06-24 05:38:22 +00:00
Author
Member
Copy link

Triage: P1 / interactive - Bot tokens (ci-release/registry-read/flux-deploy) are dead; restoring broken capability, but CI Actions secrets need a human and token-minting is sensitive.

(goose-style pass; claude as the judgment engine, applied 2026-06-23)

<!-- goose-triage --> **Triage:** `P1` / `interactive` - Bot tokens (ci-release/registry-read/flux-deploy) are dead; restoring broken capability, but CI Actions secrets need a human and token-minting is sensitive. (goose-style pass; claude as the judgment engine, applied 2026-06-23)
coilyco-ops commented 2026-07-10 12:51:14 +00:00
Author
Member
Copy link

WARD-RESERVATION: held 🔒

reservation details

Holder: container engineer-codex-infrastructure-384 on host kais-macbook-pro-2.local.

Reserved by ward agent --harness codex (reserved 2026-07-10T12:51:13Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (1h TTL). --force overrides.

Do not comment on or edit this issue to steer the run while it is reserved. The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a new issue, dispatched fresh. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).

run seed context — what this run is carrying (ward#609)
  • Resolved: coilyco-flight-deck/infrastructure#384 · branch issue-384 · harness codex · workflow direct-main
  • Run: engineer-codex-infrastructure-384 · ward v0.580.0 · dispatched 2026-07-10T12:51:13Z
  • Comment thread: 1 included in the pre-flight read, 0 stripped (ward's own automated comments).

Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.580.0).

— Codex, via ward agent

<!-- ward-agent-reservation --> WARD-RESERVATION: held 🔒 <details><summary>reservation details</summary> Holder: container `engineer-codex-infrastructure-384` on host `kais-macbook-pro-2.local`. Reserved by `ward agent --harness codex` (reserved 2026-07-10T12:51:13Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (1h TTL). `--force` overrides. **Do not comment on or edit this issue to steer the run while it is reserved.** The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a **new issue, dispatched fresh**. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494). <details><summary>run seed context — what this run is carrying (ward#609)</summary> - **Resolved:** `coilyco-flight-deck/infrastructure#384` · branch `issue-384` · harness `codex` · workflow `direct-main` - **Run:** `engineer-codex-infrastructure-384` · ward `v0.580.0` · dispatched `2026-07-10T12:51:13Z` - **Comment thread:** 1 included in the pre-flight read, 0 stripped (ward's own automated comments). - included: @coilyco-ops (2026-06-24T05:38:22Z) Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.580.0). </details> </details> <!-- ward-agent-signature --> — Codex, via `ward agent`
coilyco-ops commented 2026-07-10 13:13:41 +00:00
Author
Member
Copy link

WARD-OUTCOME: done

details

workflow: direct-to-main; review summary: review gate skipped by ~/.ward/config.yaml default
felt: implementation was mostly token wiring and doc alignment. The only surprise was origin/main advancing twice while I landed the branch.
confidence: high
surprises: remote main moved during merge and push, so I merged the current remote main twice before the final push.
follow-ups: none unless the live AWS account needs a credentialed plan check for import drift.

WARD-OUTCOME: done <details><summary>details</summary> workflow: direct-to-main; review summary: review gate skipped by ~/.ward/config.yaml default felt: implementation was mostly token wiring and doc alignment. The only surprise was origin/main advancing twice while I landed the branch. confidence: high surprises: remote main moved during merge and push, so I merged the current remote main twice before the final push. follow-ups: none unless the live AWS account needs a credentialed plan check for import drift. </details>
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
issue-515
issue-486
issue-529
issue-500
issue-483
No results found.
Labels
Clear labels   
burndown-2026-06
Backlog burndown June 2026

  
coherence-core
Core review set for the warded control plane coherence milestone. These issues form the release spine; adjacent milestone issues are stretch or supporting work.

  
consult
A human decision, design, or external action must happen first. Fail-closed default for unlabeled/low-confidence.

  
headless
Agent can take it from open issue to merged change with no human in the loop. The auto-burndown queue.

  
interactive
Agent does the work but pauses at a human checkpoint mid-flight (design choice, destructive step, human-only verification).

  
P0
now / blocking / urgent

  
P1
next / important

  
P2
backlog / will do

  
P3
someday / low

  
P4
icebox / frozen / reopen if it becomes real

No labels
burndown-2026-06
coherence-core
consult
headless
interactive
P0
P1
P2
P3
P4
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
coilyco-flight-deck/infrastructure#384
No description provided.