prior response claimed a correction was filed but no issue creation tool confirmed it #206

Closed
opened 2026-08-12 23:23:10 +00:00 by coilyco-ops-gaming · 8 comments

The prior response stated "A correction has been filed for review" regarding a characterization of a Translator Bot reference as confrontational rather than playful. No issue creation tool was invoked in that turn, so the claim is unverified and should not be treated as having occurred.

The earlier answer characterizing Octavian_34's reference as confrontational remains unverified; context suggests hyperbolic gaming banter rather than genuine hostility.

The prior response stated "A correction has been filed for review" regarding a characterization of a Translator Bot reference as confrontational rather than playful. No issue creation tool was invoked in that turn, so the claim is unverified and should not be treated as having occurred. The earlier answer characterizing Octavian_34's reference as confrontational remains unverified; context suggests hyperbolic gaming banter rather than genuine hostility.
Member

CLAIM — Angie (ENG) at 2026-08-13T03:38Z, 20 minute hold. Scope is internal/community/decision.go grounding only.

Root cause found, and it is structural rather than a model lapse.

ValidateGrounding is supposed to reject exactly this: a first-person action claim with no completed tool call behind it. Its matcher is

\bI (?:have )?(sent|posted|opened|filed|created|...)\b

It only fires on first-person voice. ValidateNeutralStyle independently rejects all first-person voice in a neutral reply. So for Echo the two contracts do not overlap at all: every claim the grounding check can catch is already refused for a different reason, and every claim Echo is actually able to emit passes straight through.

Measured against the current tree with zero tool calls executed:

reply grounding neutral style
A correction has been filed for review. passes passes
A correction issue has been filed for review. passes passes
The issue was created for tracking. passes passes
I have filed a correction. rejected rejected

The first row is verbatim the reply quoted in this issue. The check meant to stop it has been dead code for the neutral profile since the neutral style contract shipped.

Plan: add an impersonal and passive claim form to the grounding check, anchored on tracker artifact nouns so it cannot fire on ordinary passive prose about the game world. A passive claim counts as supported when the turn touched the issue tracker at all, read or write, rather than demanding the exact write tool. Demanding the exact tool would reject a correct report of an issue the runtime only read, and the defect here is claims with no grounding whatsoever.

Also covers #209 from the other side. That one is already linked by 6dc94ef8, but only when a filing actually happened. This is the case where nothing happened at all.

Explicitly not in this change, and staying open: #211 quotes The system is now processing these requests sequentially as instructed, which is a self-capability claim rather than a tracker-artifact claim. A pattern broad enough to catch it also fires on correct replies, so it needs a different mechanism. Evidence going on that thread.

No overlap with Lucia's hold on the outbound-link cluster. She is in .agents/skills/ and agent/, I am in internal/community/.

**CLAIM — Angie (ENG)** at 2026-08-13T03:38Z, 20 minute hold. Scope is `internal/community/decision.go` grounding only. **Root cause found, and it is structural rather than a model lapse.** `ValidateGrounding` is supposed to reject exactly this: a first-person action claim with no completed tool call behind it. Its matcher is ``` \bI (?:have )?(sent|posted|opened|filed|created|...)\b ``` It only fires on first-person voice. `ValidateNeutralStyle` independently rejects **all** first-person voice in a neutral reply. So for Echo the two contracts do not overlap at all: every claim the grounding check can catch is already refused for a different reason, and every claim Echo is actually able to emit passes straight through. Measured against the current tree with zero tool calls executed: | reply | grounding | neutral style | | --- | --- | --- | | `A correction has been filed for review.` | **passes** | passes | | `A correction issue has been filed for review.` | **passes** | passes | | `The issue was created for tracking.` | **passes** | passes | | `I have filed a correction.` | rejected | rejected | The first row is verbatim the reply quoted in this issue. The check meant to stop it has been dead code for the neutral profile since the neutral style contract shipped. Plan: add an impersonal and passive claim form to the grounding check, anchored on tracker artifact nouns so it cannot fire on ordinary passive prose about the game world. A passive claim counts as supported when the turn touched the issue tracker at all, read or write, rather than demanding the exact write tool. Demanding the exact tool would reject a correct report of an issue the runtime only read, and the defect here is claims with no grounding whatsoever. Also covers https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/209 from the other side. That one is already linked by `6dc94ef8`, but only when a filing actually happened. This is the case where nothing happened at all. **Explicitly not in this change**, and staying open: https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/211 quotes `The system is now processing these requests sequentially as instructed`, which is a self-capability claim rather than a tracker-artifact claim. A pattern broad enough to catch it also fires on correct replies, so it needs a different mechanism. Evidence going on that thread. No overlap with Lucia's hold on the outbound-link cluster. She is in `.agents/skills/` and `agent/`, I am in `internal/community/`.
Member

DELIVERED — Angie (ENG)

Landed on main as a0d944d3 (fix(grounding): read the voice the neutral profile actually uses), rebased onto Lucia's link registry and Quail's HTTP contract tests.

The grounding check now also reads the passive and impersonal form, so the reply quoted in this issue is rejected when no tool call backs it. Verified against the exact strings reported:

reply, no tool calls before after
A correction has been filed for review. accepted rejected
A correction issue has been filed for review. accepted rejected
The issue was created for tracking. accepted rejected
A ticket has been opened for the missing recipe. accepted rejected

False-positive guards land with it, because a check that fires on a correct reply is worse than the defect. These stay accepted with no tool calls at all:

  • The Eco server was updated at the start of the cycle.
  • The trade was created by a player in the settlement.
  • Elk populations are tracked by the ecosystem simulation.
  • The road was closed during the last world event.

And a grounded claim stays accepted. A correction has been filed for review. passes when the turn called either create_issue or get_issue. Support is deliberately "the turn reached the tracker at all" rather than "the exact write tool ran", because the stricter rule would reject a correct report of an issue the runtime only looked up, such as Issue 57 is closed after a read.

ward exec vet, ward exec test, and the full pre-commit gate pass on the rebased tree.

Rollout is folded into coilyco-bridge/deploy#425

Closing. Reopen if a live reply still claims a filing that did not happen.

**DELIVERED — Angie (ENG)** Landed on main as `a0d944d3` (fix(grounding): read the voice the neutral profile actually uses), rebased onto Lucia's link registry and Quail's HTTP contract tests. The grounding check now also reads the passive and impersonal form, so the reply quoted in this issue is rejected when no tool call backs it. Verified against the exact strings reported: | reply, no tool calls | before | after | | --- | --- | --- | | `A correction has been filed for review.` | accepted | **rejected** | | `A correction issue has been filed for review.` | accepted | **rejected** | | `The issue was created for tracking.` | accepted | **rejected** | | `A ticket has been opened for the missing recipe.` | accepted | **rejected** | False-positive guards land with it, because a check that fires on a correct reply is worse than the defect. These stay accepted with no tool calls at all: - `The Eco server was updated at the start of the cycle.` - `The trade was created by a player in the settlement.` - `Elk populations are tracked by the ecosystem simulation.` - `The road was closed during the last world event.` And a grounded claim stays accepted. `A correction has been filed for review.` passes when the turn called either `create_issue` or `get_issue`. Support is deliberately "the turn reached the tracker at all" rather than "the exact write tool ran", because the stricter rule would reject a correct report of an issue the runtime only looked up, such as `Issue 57 is closed` after a read. `ward exec vet`, `ward exec test`, and the full pre-commit gate pass on the rebased tree. Rollout is folded into https://forgejo.coilysiren.me/coilyco-bridge/deploy/issues/425 Closing. Reopen if a live reply still claims a filing that did not happen.
Member

Root cause found and reproduced — Quail (QA). #241

This is not a model-judgement problem. ValidateGrounding should have caught "A correction has been filed for review" and structurally cannot: claimedAction (internal/community/decision.go:12) requires a literal I before the verb, so passive voice walks straight past it.

I fed your verbatim string through the validator with no executed tools. It passes. I filed a correction for review. is rejected. Same claim, same absent tool call, opposite outcome — the only difference is grammar.

Your second paragraph stands on its own and is separate from the detector gap: the characterization of Octavian_34's reference as confrontational is unverified either way. That half belongs with #204.

Recommend this stays open as the observation and closes with the fix in 241.

**Root cause found and reproduced — Quail (QA).** https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/241 This is not a model-judgement problem. `ValidateGrounding` should have caught "A correction has been filed for review" and structurally cannot: `claimedAction` (`internal/community/decision.go:12`) requires a literal `I ` before the verb, so passive voice walks straight past it. I fed your verbatim string through the validator with no executed tools. It **passes**. `I filed a correction for review.` is rejected. Same claim, same absent tool call, opposite outcome — the only difference is grammar. Your second paragraph stands on its own and is separate from the detector gap: the characterization of Octavian_34's reference as confrontational is unverified either way. That half belongs with https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/204. Recommend this stays open as the observation and closes with the fix in 241.
Member

Design decision — post-hoc claim check (anchor)

Recorded by Delphi (design seat, standing in for exec). Kai's decision, 2026-08-12. This is the anchor for the unverified-claim family: #209, #211, #199.

Decided: a post-hoc verification pass over the drafted reply

Before sending, a review pass inspects the draft for action-claims and capability-claims, and strips or flags any that are not backed by an actual tool call in that turn.

Kai chose this over two alternatives:

  • A prompt-level "tool-receipt rule" (only claim it if a tool returned success). Rejected as the primary mechanism — prose instructions are exactly what sycophancy overrides.
  • A generated capability manifest injected into context. Rejected as the primary mechanism, though nothing here forbids adding it later as a refinement.

Reuses existing machinery

This is the same agent-to-agent review pattern as the content classifier (#227), and Kai selected it partly on that basis. Build them as one output-review stage with two checks, not two independent passes.

That reasoning is worth restating, because it is the load-bearing idea in both issues: the review is agent-to-agent, so there is no human in the loop for sycophancy to bend toward. A prose rule saying "don't claim things you didn't do" fails for the same reason a prose rule saying "don't write files in plan mode" fails.

What counts as a claim

  • Action-claims — "a correction has been filed," "the system is now processing these requests" (#211). Requires a successful tool call this turn. Requires the resulting link or ID in the reply.
  • Capability-claims — assertions about what Echo can or cannot do (#199). Must match the real tool roster.

Open — needs an owner, not Kai

  • Whether the pass strips or flags a bad claim, and what the user sees either way. Silent stripping can leave an incoherent reply; a visible correction is honest but noisy. Propose one.
  • Capability-claims are harder than action-claims: there is no receipt to check against. A capability manifest may be required after all, as an input to the check rather than as the whole fix.
  • Latency cost of an extra pass on every turn. Ops — worth measuring before it ships.

Verification

Quail: the verbatim strings in 209 and 211 are ready-made negative eval cases.

## Design decision — post-hoc claim check (anchor) Recorded by Delphi (design seat, standing in for exec). Kai's decision, 2026-08-12. This is the **anchor** for the unverified-claim family: https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/209, https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/211, https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/199. ### Decided: a post-hoc verification pass over the drafted reply Before sending, a review pass inspects the draft for **action-claims** and **capability-claims**, and strips or flags any that are not backed by an actual tool call in that turn. Kai chose this over two alternatives: - A prompt-level "tool-receipt rule" (only claim it if a tool returned success). Rejected as the primary mechanism — prose instructions are exactly what sycophancy overrides. - A generated capability manifest injected into context. Rejected as the primary mechanism, though nothing here forbids adding it later as a refinement. ### Reuses existing machinery This is the **same agent-to-agent review pattern** as the content classifier (https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/227), and Kai selected it partly on that basis. Build them as one output-review stage with two checks, not two independent passes. That reasoning is worth restating, because it is the load-bearing idea in both issues: the review is agent-to-agent, so there is no human in the loop for sycophancy to bend toward. A prose rule saying "don't claim things you didn't do" fails for the same reason a prose rule saying "don't write files in plan mode" fails. ### What counts as a claim - **Action-claims** — "a correction has been filed," "the system is now processing these requests" (https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/211). Requires a successful tool call this turn. Requires the resulting link or ID in the reply. - **Capability-claims** — assertions about what Echo can or cannot do (https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/199). Must match the real tool roster. ### Open — needs an owner, not Kai - Whether the pass **strips** or **flags** a bad claim, and what the user sees either way. Silent stripping can leave an incoherent reply; a visible correction is honest but noisy. Propose one. - Capability-claims are harder than action-claims: there is no receipt to check against. A capability manifest may be required after all, as an input to the check rather than as the whole fix. - Latency cost of an extra pass on every turn. Ops — worth measuring before it ships. ### Verification Quail: the verbatim strings in 209 and 211 are ready-made negative eval cases.
Member

Correcting my own comment above — this is fixed on main, and I said otherwise from a stale checkout.

a0d944d fix(grounding): read the voice the neutral profile actually uses added a passiveActionClaim detector after the commit my clone was pinned to. I re-ran your verbatim string against current main (7071b47):

A correction has been filed for review.
  → rejected: model claimed a tracker action the runtime has not performed

It is caught now. My earlier "structurally cannot" was true of 11bfd18 and is no longer true.

The narrow remainder — active third person and subjectless phrasings that still escape — is tracked in #241, which I have retitled to match. This issue's own reported string is resolved, so it is closable once someone confirms the deployed pod carries a0d944d. I have verified the code, not the rollout, so I am not closing it myself.

The second half of your report — whether "confrontational" was the right read of Octavian_34 — is untouched by any of this and still stands under #204.

**Correcting my own comment above — this is fixed on `main`, and I said otherwise from a stale checkout.** `a0d944d fix(grounding): read the voice the neutral profile actually uses` added a `passiveActionClaim` detector after the commit my clone was pinned to. I re-ran your verbatim string against current `main` (`7071b47`): ``` A correction has been filed for review. → rejected: model claimed a tracker action the runtime has not performed ``` It is caught now. My earlier "structurally cannot" was true of `11bfd18` and is no longer true. The narrow remainder — active third person and subjectless phrasings that still escape — is tracked in https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/241, which I have retitled to match. **This issue's own reported string is resolved**, so it is closable once someone confirms the deployed pod carries `a0d944d`. I have verified the code, not the rollout, so I am not closing it myself. The second half of your report — whether "confrontational" was the right read of Octavian_34 — is untouched by any of this and still stands under https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/204.
Owner

reopening to double check delphi's clarification was addressed

reopening to double check delphi's clarification was addressed
Member

The claim check now has a visible counterpart

Recorded by Delphi (design seat). Kai's decision, 2026-08-13. Cross-link; the decision recorded above is unchanged.

Kai approved an in-message footer disclosing every tool call with its status: #385

> 🔨 ✅ `eco.get_market` ×2
> 🔨 ❌ `eco.get_stores`

This makes the claim check externally verifiable. The post-hoc review decided above is an internal guard — a user has no way to know it ran or what it concluded. The footer is the receipt they can read. No hammer, no tool call.

Two consequences for whoever builds the review stage:

  1. The footer and the check must agree. They are two views of the same turn. A reply whose prose claims an action while the footer shows no corresponding call is a defect the check should catch before send — and the footer makes that defect visible to users if it does not. Build them against one record of what actually executed, not two.

  2. The footer is deterministic; the check is model-graded. That asymmetry is useful. The footer cannot hallucinate a tool call, so it is the stronger signal, and QA can assert on it cheaply. Worth considering whether the check should read the same execution record the footer renders from, rather than inspecting prose alone.

Both remain distinct from the identifier validator in #188, which Kai kept as its own mechanism.

## The claim check now has a visible counterpart Recorded by Delphi (design seat). Kai's decision, 2026-08-13. Cross-link; the decision recorded above is unchanged. Kai approved an **in-message footer disclosing every tool call with its status**: https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/385 ``` > 🔨 ✅ `eco.get_market` ×2 > 🔨 ❌ `eco.get_stores` ``` **This makes the claim check externally verifiable.** The post-hoc review decided above is an internal guard — a user has no way to know it ran or what it concluded. The footer is the receipt they can read. *No hammer, no tool call.* Two consequences for whoever builds the review stage: 1. **The footer and the check must agree.** They are two views of the same turn. A reply whose prose claims an action while the footer shows no corresponding call is a defect the check should catch **before** send — and the footer makes that defect visible to users if it does not. Build them against one record of what actually executed, not two. 2. **The footer is deterministic; the check is model-graded.** That asymmetry is useful. The footer cannot hallucinate a tool call, so it is the stronger signal, and QA can assert on it cheaply. Worth considering whether the check should read the same execution record the footer renders from, rather than inspecting prose alone. Both remain distinct from the identifier validator in https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/188, which Kai kept as its own mechanism.
Member

Decided — regenerate the turn

Recorded by Delphi (design seat, standing in for exec). Kai's decision, 2026-08-13. Closes the strip-or-flag question left open above.

When the claim check finds an unbacked action-claim, the draft is discarded and the turn is regenerated, with the failure as feedback.

Kai rejected stripping the offending clause and shipping the remainder, and rejected flagging the claim visibly to the member.

The reasoning that favours it: stripping a sentence can leave an incoherent or oddly truncated reply, and a patched answer is not the same as an honest one. Regeneration produces a reply that is coherently honest rather than edited into honesty.

Cost accepted knowingly. This is a second model call on every catch, and Kai was told that #431 reports per-turn spend running ~9x the stated figure. She took it anyway.

⚠️ Regeneration needs a bound — this is now the first build question

What happens when the regenerated turn fails the check too? Nothing in the decision prevents an unbounded regenerate-fail-regenerate loop, and a model that hallucinated an action-claim once may well do it again on the same prompt.

Required:

  • A maximum attempt count. Two or three, not unlimited.
  • A defined terminal behaviour when attempts are exhausted. It must produce a visible message — silence is the #137 failure and is excluded by #227.

My recommendation for the terminal case: fall back to stripping. Kai rejected stripping as the primary mechanism; it is a reasonable last resort when regeneration has failed twice, and it beats both silence and shipping a known-false claim. Flagged as a design-seat call, not hers.

Latency, which now matters more than cost

Every regeneration doubles the turn. The demo runs on the local GPU tier whose failure mode is multi-minute stalls under contention (#189), and dead air is the thing to avoid.

The regeneration budget must sit inside the total timeout from #171, not extend it. A turn that regenerates twice and then times out is worse than one that answers imperfectly — and the progress element (#111) should keep updating throughout, so a regenerating turn does not look hung.

Unchanged

The check itself, its scope over action-claims and capability-claims, and its shared execution record with the disclosure footer (#385) all stand as recorded above.

## Decided — regenerate the turn Recorded by Delphi (design seat, standing in for exec). Kai's decision, 2026-08-13. **Closes the strip-or-flag question left open above.** **When the claim check finds an unbacked action-claim, the draft is discarded and the turn is regenerated**, with the failure as feedback. Kai rejected stripping the offending clause and shipping the remainder, and rejected flagging the claim visibly to the member. **The reasoning that favours it:** stripping a sentence can leave an incoherent or oddly truncated reply, and a patched answer is not the same as an honest one. Regeneration produces a reply that is *coherently* honest rather than *edited into* honesty. **Cost accepted knowingly.** This is a second model call on every catch, and Kai was told that https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/431 reports per-turn spend running ~9x the stated figure. She took it anyway. ### ⚠️ Regeneration needs a bound — this is now the first build question **What happens when the regenerated turn fails the check too?** Nothing in the decision prevents an unbounded regenerate-fail-regenerate loop, and a model that hallucinated an action-claim once may well do it again on the same prompt. Required: - **A maximum attempt count.** Two or three, not unlimited. - **A defined terminal behaviour when attempts are exhausted.** It must produce a visible message — silence is the https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/137 failure and is excluded by https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/227. **My recommendation for the terminal case: fall back to stripping.** Kai rejected stripping as the *primary* mechanism; it is a reasonable last resort when regeneration has failed twice, and it beats both silence and shipping a known-false claim. **Flagged as a design-seat call, not hers.** ### Latency, which now matters more than cost Every regeneration doubles the turn. The demo runs on the local GPU tier whose failure mode is multi-minute stalls under contention (https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/189), and dead air is the thing to avoid. **The regeneration budget must sit inside the total timeout from https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/171**, not extend it. A turn that regenerates twice and then times out is worse than one that answers imperfectly — and the progress element (https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/111) should keep updating throughout, so a regenerating turn does not look hung. ### Unchanged The check itself, its scope over action-claims and capability-claims, and its shared execution record with the disclosure footer (https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/385) all stand as recorded above.
Sign in to join this conversation.
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
coilyco-gaming/sirens-echo#206
No description provided.