Job record: make a unit of work first-class, durable, and idempotent #143

Closed
opened 2026-08-12 11:10:38 +00:00 by coilyco-ops · 4 comments
Member

Outcome

A job is a first-class object in the harness: it has an id, an owner, a state, and a lifecycle that outlives the turn that created it.

This is the foundation unit. Every other unit in this batch depends on it, and most of them collapse into small changes once it exists.

Why this is the first one

Today the harness has turns and it has side effects, and nothing in between. A Discord message starts a turn, the turn may call a tool, the turn ends. If the pod restarts mid-turn the work is simply gone, and nothing can be asked about afterwards.

That gap is what separates a chat agent from a platform. Almost every "platform" feature people want - progress, cancellation, resumption, per-run telemetry, addressable history - is a property of a durable work object, not a property of a conversation.

Scope

Item 1 - durable work items.

  • A job record with, at minimum: id, created-at, requesting principal, transport and origin (Discord channel or message, or /v1/turn), current state, terminal outcome, and error.
  • States named explicitly and transitions enumerated. A state machine, not a status string.
  • Persistence that survives pod restart. The deployment surface for this is tracked separately in coilyco-bridge/deploy (linked below), so this unit should treat the store as an interface rather than picking a backend inline.
  • Jobs are queryable by id and listable by requesting principal.

Item 3 - resumability and idempotency.

  • An idempotency key on job submission, so a retried summon does not create a second job.
  • Restart behavior defined per state. A job interrupted mid-execution must not silently double-apply an effect it already performed.
  • At-least-once delivery assumed from the transport, so the harness owns dedup.

Explicitly out of scope

  • Async submit/poll and cancellation - separate unit.
  • Any execution surface - separate unit.
  • Per-requester authority and attribution. Not approved in this batch. The job record should carry the requesting principal as a field regardless, because retrofitting an owner onto existing records is worse than carrying one from the start. Storing it grants nothing.

Acceptance

  • A job survives a pod restart and is retrievable by id afterward with its state intact.
  • Submitting the same request twice with the same idempotency key yields one job.
  • Every job carries the principal that requested it.
  • The state machine is documented, and an invalid transition is an error rather than a silent overwrite.
  • Deployment surface: the persistent job store, tracked in coilyco-bridge/deploy.
  • Depends on nothing. Blocks every other unit in this batch.

Next owner

Engineer.

## Outcome A job is a first-class object in the harness: it has an id, an owner, a state, and a lifecycle that outlives the turn that created it. This is the foundation unit. Every other unit in this batch depends on it, and most of them collapse into small changes once it exists. ## Why this is the first one Today the harness has turns and it has side effects, and nothing in between. A Discord message starts a turn, the turn may call a tool, the turn ends. If the pod restarts mid-turn the work is simply gone, and nothing can be asked about afterwards. That gap is what separates a chat agent from a platform. Almost every "platform" feature people want - progress, cancellation, resumption, per-run telemetry, addressable history - is a property of a durable work object, not a property of a conversation. ## Scope **Item 1 - durable work items.** * A job record with, at minimum: id, created-at, requesting principal, transport and origin (Discord channel or message, or `/v1/turn`), current state, terminal outcome, and error. * States named explicitly and transitions enumerated. A state machine, not a status string. * Persistence that survives pod restart. The deployment surface for this is tracked separately in `coilyco-bridge/deploy` (linked below), so this unit should treat the store as an interface rather than picking a backend inline. * Jobs are queryable by id and listable by requesting principal. **Item 3 - resumability and idempotency.** * An idempotency key on job submission, so a retried summon does not create a second job. * Restart behavior defined per state. A job interrupted mid-execution must not silently double-apply an effect it already performed. * At-least-once delivery assumed from the transport, so the harness owns dedup. ## Explicitly out of scope * Async submit/poll and cancellation - separate unit. * Any execution surface - separate unit. * Per-requester authority and attribution. **Not approved in this batch.** The job record should carry the requesting principal as a field regardless, because retrofitting an owner onto existing records is worse than carrying one from the start. Storing it grants nothing. ## Acceptance * A job survives a pod restart and is retrievable by id afterward with its state intact. * Submitting the same request twice with the same idempotency key yields one job. * Every job carries the principal that requested it. * The state machine is documented, and an invalid transition is an error rather than a silent overwrite. ## Related * Deployment surface: the persistent job store, tracked in `coilyco-bridge/deploy`. * Depends on nothing. Blocks every other unit in this batch. ## Next owner Engineer.
Author
Member

This is the foundation of a seven-issue batch. Recording the full map here so this issue can act as the entry point.

Origin

An itemized comparison of what Sirens Deep is today — a conversational agent whose entrypoint is Discord — against a dev platform whose entrypoint happens to be Discord. Sixteen gaps were identified; ten were approved for scoping.

Approved: durable work items, async execution, resumability/idempotency, cancellation, sandboxed execution, workspace, job-scoped telemetry, progress reporting, structured commands, thread-as-work-context.

Not approved, deliberately deferred: per-requester authority, attribution, authorization beyond admission, approval gates, per-task secret brokering, queue and backpressure.

The batch

Harness — coilyco-gaming/sirens-echo

Unit Covers
this issue job record, idempotency
#144 async submit/poll/notify, cancellation
#145 Ward-backed execution, per-job workspace
#146 job-scoped telemetry, progress reporting
#147 structured commands, thread-to-job binding

Deployment — coilyco-bridge/deploy

Unit Enables
coilyco-bridge/deploy#391 persistent job store — this issue
coilyco-bridge/deploy#392 execution workspace, Ward reach — issue 145

Dependency shape

Everything depends on this issue and nothing else depends on anything else. 144, 145, 146, and 147 are independent of each other and can proceed in parallel once the job record exists.

That is not an accident of how the batch was cut. Nine of the sixteen original gaps turned out to be the same missing thing — there is no object between "a message" and "an effect." Progress, cancellation, resumption, per-run telemetry, and addressable history are all properties of a durable work object. Build the object and most of them become small.

Sequencing note

#145 and coilyco-bridge/deploy#392 add an execution surface while per-requester authority and attribution stay deferred. Every job will act as one identity with no audit distinction between requesters.

That is sound while admission is a direct-message allowlist of one account. coilyco-bridge/deploy#365 proposes opening Sirens Deep to a Discord guild on 2026-08-19. Those two changes should not land in the same window; if the guild opens first, per-requester authority should precede execution. Both execution issues carry this note.

Batched by ops (Olaf).

This is the foundation of a seven-issue batch. Recording the full map here so this issue can act as the entry point. ## Origin An itemized comparison of what Sirens Deep is today — a conversational agent whose entrypoint is Discord — against a dev platform whose entrypoint happens to be Discord. Sixteen gaps were identified; ten were approved for scoping. Approved: durable work items, async execution, resumability/idempotency, cancellation, sandboxed execution, workspace, job-scoped telemetry, progress reporting, structured commands, thread-as-work-context. **Not approved, deliberately deferred:** per-requester authority, attribution, authorization beyond admission, approval gates, per-task secret brokering, queue and backpressure. ## The batch **Harness — `coilyco-gaming/sirens-echo`** | Unit | Covers | | --- | --- | | **this issue** | job record, idempotency | | https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/144 | async submit/poll/notify, cancellation | | https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/145 | Ward-backed execution, per-job workspace | | https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/146 | job-scoped telemetry, progress reporting | | https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/147 | structured commands, thread-to-job binding | **Deployment — `coilyco-bridge/deploy`** | Unit | Enables | | --- | --- | | https://forgejo.coilysiren.me/coilyco-bridge/deploy/issues/391 | persistent job store — this issue | | https://forgejo.coilysiren.me/coilyco-bridge/deploy/issues/392 | execution workspace, Ward reach — issue 145 | ## Dependency shape Everything depends on this issue and nothing else depends on anything else. 144, 145, 146, and 147 are independent of each other and can proceed in parallel once the job record exists. That is not an accident of how the batch was cut. Nine of the sixteen original gaps turned out to be the same missing thing — there is no object between "a message" and "an effect." Progress, cancellation, resumption, per-run telemetry, and addressable history are all properties of a durable work object. Build the object and most of them become small. ## Sequencing note https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/145 and https://forgejo.coilysiren.me/coilyco-bridge/deploy/issues/392 add an execution surface while per-requester authority and attribution stay deferred. Every job will act as one identity with no audit distinction between requesters. That is sound while admission is a direct-message allowlist of one account. https://forgejo.coilysiren.me/coilyco-bridge/deploy/issues/365 proposes opening Sirens Deep to a Discord guild on 2026-08-19. Those two changes should not land in the same window; if the guild opens first, per-requester authority should precede execution. Both execution issues carry this note. Batched by ops (Olaf).
Author
Member

Direction: the store is decided, and so is the timing

Direction from Kai, 2026-08-12 session. This is the batch entry point, so the batch-wide decisions are recorded here and referenced from the other four.

Timing

The demo track owns the week to August 19. This batch starts August 20 and is the priority from then on.

"The batch is the real work" describes where the value is; August 19 is a hard date with a permanent public recording behind it. Both are true and they do not compete for the same seven days.

Ownership: one seat (claude) carries the whole portfolio, demo track and batch alike.

The persistent store: Postgres

The issue says to treat the store as an interface rather than picking a backend inline, with the choice tracked in coilyco-bridge/deploy#391.

That choice is made: Postgres, and it is already running. Build against it directly. No interface indirection is needed for the purpose of deferring a backend decision, because there is no longer a decision to defer.

This unblocks the acceptance criterion that a job survives a pod restart and is retrievable by id afterward with its state intact — it is testable from day one rather than pending another repository's ticket.

Items 5 and 6 are un-deferred

The batch comment lists per-requester authority and attribution under not approved, deliberately deferred. Kai has approved both as scope.

They are not yet issues. The engineer files them as two separate issues in this repository, written to this batch's format — Outcome, Why, Scope, Acceptance, Depends on — and positioned as blockers on #145, per the sequencing decision recorded there.

This changes the batch's dependency shape. It is no longer "everything depends on #143 and nothing else depends on anything else": #145 now depends on #143 and on both new issues.

Unchanged in this issue

Everything else stands. In particular the point that the job record carries the requesting principal as a field regardless — that was correct when authority was deferred and it is still correct now that it is not. Storing it grants nothing; retrofitting an owner onto existing records is worse than carrying one from the start.

The state machine, the idempotency key, and per-state restart behavior are all unchanged.

## Direction: the store is decided, and so is the timing Direction from Kai, 2026-08-12 session. This is the batch entry point, so the batch-wide decisions are recorded here and referenced from the other four. ## Timing **The demo track owns the week to August 19. This batch starts August 20 and is the priority from then on.** "The batch is the real work" describes where the value is; August 19 is a hard date with a permanent public recording behind it. Both are true and they do not compete for the same seven days. Ownership: one seat (claude) carries the whole portfolio, demo track and batch alike. ## The persistent store: Postgres The issue says to treat the store as an interface rather than picking a backend inline, with the choice tracked in `coilyco-bridge/deploy#391`. **That choice is made: Postgres, and it is already running.** Build against it directly. No interface indirection is needed for the purpose of deferring a backend decision, because there is no longer a decision to defer. This unblocks the acceptance criterion that a job survives a pod restart and is retrievable by id afterward with its state intact — it is testable from day one rather than pending another repository's ticket. ## Items 5 and 6 are un-deferred The batch comment lists per-requester authority and attribution under **not approved, deliberately deferred**. **Kai has approved both as scope.** They are not yet issues. The engineer files them as **two separate issues** in this repository, written to this batch's format — Outcome, Why, Scope, Acceptance, Depends on — and positioned as **blockers on #145**, per the sequencing decision recorded there. This changes the batch's dependency shape. It is no longer "everything depends on #143 and nothing else depends on anything else": #145 now depends on #143 *and* on both new issues. ## Unchanged in this issue Everything else stands. In particular the point that the job record carries the requesting principal as a field regardless — that was correct when authority was deferred and it is still correct now that it is not. Storing it grants nothing; retrofitting an owner onto existing records is worse than carrying one from the start. The state machine, the idempotency key, and per-state restart behavior are all unchanged.
Author
Member

Items 5 and 6 are filed — updated batch map

Following the decision above, the two un-deferred items now exist as issues. Updating the batch map on this entry point so it stays the single place to read the shape.

Harness — coilyco-gaming/sirens-echo

Unit Covers
#143 (this issue) job record, idempotency
#144 async submit/poll/notify, cancellation
#145 Ward-backed execution, per-job workspace
#146 job-scoped telemetry, progress reporting
#147 structured commands, thread-to-job binding
#150 per-requester authority — item 5, newly approved
#151 attribution — item 6, newly approved

Deployment — coilyco-bridge/deploy

Unit Enables
deploy#391 persistent job store — superseded in effect: Postgres is chosen and already running
deploy#392 execution workspace, Ward reach — #145

Revised dependency shape

The original comment's "everything depends on #143 and nothing else depends on anything else" no longer holds. Current shape:

  • #143 depends on nothing. Blocks everything.
  • #144, #146, #150, #151, and the thread-binding half of #147 all depend on #143 alone and can proceed in parallel once it lands.
  • #145 depends on #143, #150, and #151. It is last, deliberately.
  • The structured-command half of #147 additionally waits on the applications.commands scope decision deferred at #127.

#151 before #150 is the recommended order within the pair; see the sequencing note there.

Still not approved

Items 7 and 8 — authorization beyond admission, approval gates — plus per-task secret brokering and queue/backpressure. Unchanged from the original batch. Kai un-deferred items 5 and 6 only, and both new issues state that boundary so neither expands into it.

## Items 5 and 6 are filed — updated batch map Following the decision above, the two un-deferred items now exist as issues. Updating the batch map on this entry point so it stays the single place to read the shape. **Harness — `coilyco-gaming/sirens-echo`** | Unit | Covers | | --- | --- | | **#143** (this issue) | job record, idempotency | | #144 | async submit/poll/notify, cancellation | | #145 | Ward-backed execution, per-job workspace | | #146 | job-scoped telemetry, progress reporting | | #147 | structured commands, thread-to-job binding | | **#150** | per-requester authority — item 5, newly approved | | **#151** | attribution — item 6, newly approved | **Deployment — `coilyco-bridge/deploy`** | Unit | Enables | | --- | --- | | deploy#391 | persistent job store — **superseded in effect**: Postgres is chosen and already running | | deploy#392 | execution workspace, Ward reach — #145 | ## Revised dependency shape The original comment's "everything depends on #143 and nothing else depends on anything else" no longer holds. Current shape: * **#143** depends on nothing. Blocks everything. * **#144**, **#146**, **#150**, **#151**, and the thread-binding half of **#147** all depend on #143 alone and can proceed in parallel once it lands. * **#145** depends on #143, **#150**, and **#151**. It is last, deliberately. * The structured-command half of **#147** additionally waits on the `applications.commands` scope decision deferred at #127. `#151` before `#150` is the recommended order within the pair; see the sequencing note there. ## Still not approved Items 7 and 8 — authorization beyond admission, approval gates — plus per-task secret brokering and queue/backpressure. Unchanged from the original batch. Kai un-deferred items 5 and 6 only, and both new issues state that boundary so neither expands into it.
Author
Member

Correction: the store is provisioned, and it brings requirements with it

My earlier comment said deploy#391 was "superseded in effect." That is wrong and I am correcting it before anyone builds on it. 391 was not superseded — it was implemented, in coilyco-bridge/deploy#393, merged 2026-08-12T11:40:45Z. I was reading a checkout from 08:15 that predated the work.

The correction matters because 391 and 393 settled several things this issue now inherits rather than gets to decide.

What exists

Per lane — sirens-deep and sirens-echo, both — four objects copied from services/eco-gnome/deploy/main.yml:

  • postgres:17, one replica, strategy: Recreate
  • 2Gi local-path RWO PVC, pinned to kai-server
  • ClusterIP Service on 5432
  • ExternalSecret templating one SSM SecureString into two keys: POSTGRES_PASSWORD for the database, and DSN for the harness

Rollout ordering puts the store ahead of the app on both lanes, so the harness resolves its store at startup rather than on first write.

Not verified live. PR 393 lists what an operator still has to confirm: ExternalSecret sync, PVC binding, Postgres initialising on a volume root it does not own, and pg_isready readiness.

Four things this issue now owns

1. Name the DSN environment variable. Deliberately left to this issue. The Secret already carries the DSN key; neither values file wires it yet, so adding it is one extraEnv entry once the name exists. Ops proposed SIRENS_ECHO_JOB_STORE_DSN. Pick it or pick better, but pick — nothing connects until it is named.

The DSN is spelled as a URL rather than a keyword string, because the harness is Go. Passwords are alphanumeric by construction so no percent-encoding is involved.

2. Migrations are harness-owned and applied on boot. No migration Job exists in the deploy repo or anywhere else, by design. Copy eco-gnome's shape: apply on boot with a retry loop while Postgres warms. This is a real requirement on this issue that its body does not currently mention.

3. Echo gets a store too, so decide what Echo does with it. deploy#391 scope item 3 named the trap directly:

The two lanes share an image. If the harness gains a job store, Echo either gets one or is explicitly configured without one. A half-configured lane that starts and then fails on first write is the bad outcome.

Both stores are now provisioned. This issue has to make Echo's behaviour deliberate rather than incidental.

4. There are no backups. Recorded as a decision, not an oversight. The PVC is the only copy; a lost disk or PVC loses job history including in-flight records. Persistent is not durable here, and the state machine should not assume otherwise.

Unchanged

Postgres is still the answer and it is still already running, so this issue is not blocked. Everything above is detail arriving with it rather than a reversal.

## Correction: the store is provisioned, and it brings requirements with it My earlier comment said `deploy#391` was "superseded in effect." **That is wrong and I am correcting it before anyone builds on it.** 391 was not superseded — it was implemented, in `coilyco-bridge/deploy#393`, merged 2026-08-12T11:40:45Z. I was reading a checkout from 08:15 that predated the work. The correction matters because 391 and 393 settled several things this issue now inherits rather than gets to decide. ## What exists Per lane — `sirens-deep` **and** `sirens-echo`, both — four objects copied from `services/eco-gnome/deploy/main.yml`: * `postgres:17`, one replica, `strategy: Recreate` * 2Gi `local-path` RWO PVC, pinned to kai-server * ClusterIP Service on 5432 * ExternalSecret templating one SSM SecureString into two keys: `POSTGRES_PASSWORD` for the database, and **`DSN`** for the harness Rollout ordering puts the store ahead of the app on both lanes, so the harness resolves its store at startup rather than on first write. **Not verified live.** PR 393 lists what an operator still has to confirm: ExternalSecret sync, PVC binding, Postgres initialising on a volume root it does not own, and `pg_isready` readiness. ## Four things this issue now owns **1. Name the DSN environment variable.** Deliberately left to this issue. The Secret already carries the `DSN` key; neither values file wires it yet, so adding it is one `extraEnv` entry once the name exists. Ops proposed **`SIRENS_ECHO_JOB_STORE_DSN`**. Pick it or pick better, but pick — nothing connects until it is named. The DSN is spelled as a URL rather than a keyword string, because the harness is Go. Passwords are alphanumeric by construction so no percent-encoding is involved. **2. Migrations are harness-owned and applied on boot.** No migration Job exists in the deploy repo or anywhere else, by design. Copy eco-gnome's shape: apply on boot with a **retry loop while Postgres warms**. This is a real requirement on this issue that its body does not currently mention. **3. Echo gets a store too, so decide what Echo does with it.** `deploy#391` scope item 3 named the trap directly: > The two lanes share an image. If the harness gains a job store, Echo either gets one or is explicitly configured without one. A half-configured lane that starts and then fails on first write is the bad outcome. Both stores are now provisioned. This issue has to make Echo's behaviour deliberate rather than incidental. **4. There are no backups.** Recorded as a decision, not an oversight. The PVC is the only copy; a lost disk or PVC loses job history including in-flight records. Persistent is not durable here, and the state machine should not assume otherwise. ## Unchanged Postgres is still the answer and it is still already running, so this issue is not blocked. Everything above is detail arriving with it rather than a reversal.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
coilyco-gaming/sirens-echo#143
No description provided.