Enforce verified deploy-state transitions per environment

coilyco-ops commented

2026-07-31 08:21:28 +00:00

Member

Parent: #1625

Goal

Turn an approved release candidate into a verified, durable deploy-state transition without allowing the recorded deploy repository state and observed environment state to knowingly diverge.

This work consumes the generic composed-agent dispatch from #1626 and the Ops candidate/result contract from #1627. It does not add provider-specific deployment logic to Ward.

Workflow

For each target environment, the Ops workflow must:

Read and bind the starting deploy repository revision.
Acquire per-environment serialization or an equivalent compare-and-swap lease.
Prepare a local deploy repository change that names the immutable application revision.
Invoke repository-provided deployment operations.
Verify the target environment through repository-provided verification.
Push the deploy commit only after verification succeeds.

If deployment partially changes the environment and then fails, restore the previous committed state and verify that restoration. If the verified deploy succeeds but recording the deploy commit fails, either complete the push safely or restore and verify the previous state. The workflow must never knowingly finish with the environment and deploy repository describing different revisions.

Rollback is another verified state transition and produces its own commit.

Acceptance criteria

A successful attempt pushes exactly one verified deploy commit with application revision, environment, Ward run, and originating-ticket provenance.
A failed, rejected, or externally blocked attempt creates no deploy commit.
Concurrent attempts for one environment resolve deterministically, and a stale starting revision is rejected before mutation.
A partial deployment failure restores and verifies the previous committed state.
A verification failure cannot be recorded as a successful deploy.
A push failure cannot leave a known, unrecorded environment state.
Ward stays deployment-provider neutral. Deploy and verify behavior comes from the target repository.
Disposable fixtures cover success, concurrent conflict, partial failure, verification failure, push failure, and rollback.

Execution mode

AFK. Use disposable fixtures only. Do not deploy to or mutate live systems.

Blocked by

#1626
#1627

Parent: #1625 ## Goal Turn an approved release candidate into a verified, durable deploy-state transition without allowing the recorded deploy repository state and observed environment state to knowingly diverge. This work consumes the generic composed-agent dispatch from #1626 and the Ops candidate/result contract from #1627. It does not add provider-specific deployment logic to Ward. ## Workflow For each target environment, the Ops workflow must: 1. Read and bind the starting deploy repository revision. 2. Acquire per-environment serialization or an equivalent compare-and-swap lease. 3. Prepare a local deploy repository change that names the immutable application revision. 4. Invoke repository-provided deployment operations. 5. Verify the target environment through repository-provided verification. 6. Push the deploy commit only after verification succeeds. If deployment partially changes the environment and then fails, restore the previous committed state and verify that restoration. If the verified deploy succeeds but recording the deploy commit fails, either complete the push safely or restore and verify the previous state. The workflow must never knowingly finish with the environment and deploy repository describing different revisions. Rollback is another verified state transition and produces its own commit. ## Acceptance criteria * A successful attempt pushes exactly one verified deploy commit with application revision, environment, Ward run, and originating-ticket provenance. * A failed, rejected, or externally blocked attempt creates no deploy commit. * Concurrent attempts for one environment resolve deterministically, and a stale starting revision is rejected before mutation. * A partial deployment failure restores and verifies the previous committed state. * A verification failure cannot be recorded as a successful deploy. * A push failure cannot leave a known, unrecorded environment state. * Ward stays deployment-provider neutral. Deploy and verify behavior comes from the target repository. * Disposable fixtures cover success, concurrent conflict, partial failure, verification failure, push failure, and rollback. ## Execution mode AFK. Use disposable fixtures only. Do not deploy to or mutate live systems. ## Blocked by * #1626 * #1627

coilyco-ops referenced this issue

2026-07-31 08:21:36 +00:00

Prove issue-to-verified-deployment in a disposable fixture #1629

coilyco-ops referenced this issue

2026-07-31 08:21:52 +00:00

Add an Ops role for verified GitOps deployment handoff #1625

coilyco-ops referenced this issue

2026-07-31 08:21:55 +00:00

Generalize the dispatch broker for arbitrary composed agents #1626

coilyco-ops added the

labels

2026-07-31 08:25:56 +00:00

coilyco-ops referenced this issue

2026-07-31 08:29:07 +00:00

Add an Ops role for verified GitOps deployment handoff #1625

coilyco-ops referenced this issue

2026-07-31 09:02:20 +00:00

Add an Ops role for verified GitOps deployment handoff #1625

coilyco-ops added

and removed

labels

2026-08-05 10:26:06 +00:00

coilyco-ops referenced this issue

2026-08-05 10:39:36 +00:00

Define the Ops release-candidate and outcome contract #1627

coilyco-ops commented

2026-08-05 10:44:24 +00:00

Author

Member

Decision resolved with Kai:

#1628 is implemented as a Ward-enforced transaction state machine, not an Ops prompt convention.
Acquire a dedicated remote Git lock ref keyed by deploy repository, target branch, and environment. Creation and deletion use exact-OID compare-and-swap so serialization covers multiple Ward hosts.
Lock refs are transient coordination, not deployment state. The deploy branch remains the sole durable record of verified environment state.
Never steal a lock by TTL or delete it blindly. An existing lock produces blocked. The same attempt may resume its own lock. An indeterminate attempt retains the lock until reconciliation.
After acquiring the lock, refetch the deploy branch and require the candidate starting commit exactly. A stale candidate is rejected before mutation.
Verify the current environment against the starting deploy state before changing anything. Pre-existing drift is indeterminate and blocks deployment.
Prepare exactly one local deploy commit containing the candidate state and required provenance. Deploy that exact working state but do not push it before verification.
Once deployment begins, every error or interruption enters recovery. Ward may not simply mark the run failed and exit.
Successful verification requires a typed attestation of the exact application revision and artifact digest. Raw provider output does not enter durable artifacts.
Any failed apply or verification reapplies the starting state and verifies it against the preflight attestation. A successful recovery yields restored. Failed or uncertain recovery yields indeterminate.
After successful verification, push the prepared commit using compare-and-swap from the starting commit. Resolve an ambiguous push by rereading the remote:
- prepared commit present means verified;
- starting commit present permits bounded retry, followed by restore if retries exhaust;
- any other commit or unreadable state means indeterminate.
Release the lock only after verified, restored, or a pre-mutation terminal result. A lock-release failure leaves the Ward lifecycle at cleanup-needed.
Rollback is a new candidate targeting the prior immutable revision from the current deploy commit. It creates a new verified commit. Never rewind or force-push deploy history.
Journal every transaction phase so restart recovery resumes or restores safely.
Tests use disposable bare Git remotes and an environment simulator only. They do not add a public verification-fixture mode or touch live operations.

Decision resolved with Kai: * #1628 is implemented as a Ward-enforced transaction state machine, not an Ops prompt convention. * Acquire a dedicated remote Git lock ref keyed by deploy repository, target branch, and environment. Creation and deletion use exact-OID compare-and-swap so serialization covers multiple Ward hosts. * Lock refs are transient coordination, not deployment state. The deploy branch remains the sole durable record of verified environment state. * Never steal a lock by TTL or delete it blindly. An existing lock produces `blocked`. The same attempt may resume its own lock. An `indeterminate` attempt retains the lock until reconciliation. * After acquiring the lock, refetch the deploy branch and require the candidate starting commit exactly. A stale candidate is `rejected` before mutation. * Verify the current environment against the starting deploy state before changing anything. Pre-existing drift is `indeterminate` and blocks deployment. * Prepare exactly one local deploy commit containing the candidate state and required provenance. Deploy that exact working state but do not push it before verification. * Once deployment begins, every error or interruption enters recovery. Ward may not simply mark the run failed and exit. * Successful verification requires a typed attestation of the exact application revision and artifact digest. Raw provider output does not enter durable artifacts. * Any failed apply or verification reapplies the starting state and verifies it against the preflight attestation. A successful recovery yields `restored`. Failed or uncertain recovery yields `indeterminate`. * After successful verification, push the prepared commit using compare-and-swap from the starting commit. Resolve an ambiguous push by rereading the remote: * prepared commit present means `verified`; * starting commit present permits bounded retry, followed by restore if retries exhaust; * any other commit or unreadable state means `indeterminate`. * Release the lock only after `verified`, `restored`, or a pre-mutation terminal result. A lock-release failure leaves the Ward lifecycle at `cleanup-needed`. * Rollback is a new candidate targeting the prior immutable revision from the current deploy commit. It creates a new verified commit. Never rewind or force-push deploy history. * Journal every transaction phase so restart recovery resumes or restores safely. * Tests use disposable bare Git remotes and an environment simulator only. They do not add a public verification-fixture mode or touch live operations.

coilyco-ops referenced this issue

2026-08-05 10:47:12 +00:00

Prove issue-to-verified-deployment in a disposable fixture #1629

coilyco-ops referenced this issue

2026-08-05 10:51:04 +00:00

Remove public verification-fixture mode #1645

coilyco-ops referenced this issue from a commit

2026-08-06 04:56:57 +00:00

feat: enforce verified deploy-state transactions

coilyco-ops closed this issue

2026-08-06 04:56:57 +00:00

coilyco-ops commented

2026-08-06 04:57:12 +00:00

Author

Member

WARD-OUTCOME: done

Landed on canonical main as d04e4a87536a022524f0628abea889da20d460cb.

Evidence:

Added the provider-neutral ward agent release execute transaction with a remote per-environment Git CAS lock, exact starting-revision binding, preflight attestation, prepared-commit provenance validation, apply and verify ordering, leased push, recovery, restart reconciliation, and cleanup-needed journaling.
Verified and restored broker results now require their matching transaction outcome and typed attestation.
Disposable bare-Git and environment fixtures cover success, concurrent conflict, stale start, partial apply failure, verification failure, push failure and reconciliation, rollback, restart recovery, and failed lock cleanup.
ward exec test, lint, vet, build, test-windows-compile, test-agent-framework, test-policy-boundary, test-release-contract, lint-refs, and pre-commit passed.
ward doctor passed with only the pre-existing historical raw-archive warning.

WARD-OUTCOME: done Landed on canonical `main` as `d04e4a87536a022524f0628abea889da20d460cb`. Evidence: * Added the provider-neutral `ward agent release execute` transaction with a remote per-environment Git CAS lock, exact starting-revision binding, preflight attestation, prepared-commit provenance validation, apply and verify ordering, leased push, recovery, restart reconciliation, and cleanup-needed journaling. * Verified and restored broker results now require their matching transaction outcome and typed attestation. * Disposable bare-Git and environment fixtures cover success, concurrent conflict, stale start, partial apply failure, verification failure, push failure and reconciliation, rollback, restart recovery, and failed lock cleanup. * `ward exec test`, `lint`, `vet`, `build`, `test-windows-compile`, `test-agent-framework`, `test-policy-boundary`, `test-release-contract`, `lint-refs`, and `pre-commit` passed. * `ward doctor` passed with only the pre-existing historical raw-archive warning.

Rows
Columns

Enforce verified deploy-state transitions per environment #1628

Goal

Workflow

Acceptance criteria

Execution mode

Blocked by