feat: check a member's ticket for work and scope before filing it #928

Merged

coilysiren merged 1 commit from issue-852-filing-validity into main

2026-08-17 20:06:52 +00:00

coilyco-ops commented

2026-08-17 20:02:44 +00:00

Member

Two stages, as decided. Validity then scope, each a model call answering from a closed list.

Built against #907, not #849

The validity stage asks "is there work in this?" rather than "is this an attack", because #907 is the cleaner example: polite, on topic, well formed, and a tracker entry with no actionable content duplicating two better issues filed the same day. A check tuned for hostile input passes that straight through.

validity - work, placeholder, unclear. Refuses the last two.
scope - in-scope, out-of-scope. Refuses the second.

Three properties borrowed from the content gate

A filing check that gets these wrong is worse than no check:

A checker that fails files anyway. A dead classifier must not eat the thing the member asked for.
An answer off the closed list is the checker failing rather than refusing, so an invented verdict cannot block a member.
The refusal is harness-authored and says what would fix it, because the model reads it and relays it: "Say what should change and what is wrong with it now, and it can be filed."

Placement

The check runs in the tool-call path, beside the label injection and before it, so a refused filing is never labelled as one that happened. That seam catches every filing regardless of how the turn got there, which is the same reason the label lives there.

The principal is exempt, and the checks are audited under a harness request id rather than the requester's, so a member identifier does not reach the proxy's audit metadata.

What is unchanged

SANDBOXED, including the property that the harness applies it before dispatch so the model never supplies it and cannot omit it. The two controls answer different questions and both still run.

For #874

The two stages share one scaffolding, filingVerdict, which takes a prompt and a closed vocabulary. The citation classifier can take it as is.

Tests

The worked shape refused at stage one, an out-of-scope request refused at stage two with both stages asking different questions, a real request passing both, the principal skipping them entirely, a failed checker filing anyway, an invented verdict not refusing, and the refusal arriving as a tool result rather than failing the turn.

just gate PASS.

closes #852

Two stages, as decided. Validity then scope, each a model call answering from a closed list. ## Built against #907, not #849 The validity stage asks **"is there work in this?"** rather than "is this an attack", because #907 is the cleaner example: polite, on topic, well formed, and a tracker entry with no actionable content duplicating two better issues filed the same day. A check tuned for hostile input passes that straight through. * **validity** - `work`, `placeholder`, `unclear`. Refuses the last two. * **scope** - `in-scope`, `out-of-scope`. Refuses the second. ## Three properties borrowed from the content gate A filing check that gets these wrong is worse than no check: * **A checker that fails files anyway.** A dead classifier must not eat the thing the member asked for. * **An answer off the closed list is the checker failing rather than refusing**, so an invented verdict cannot block a member. * **The refusal is harness-authored and says what would fix it**, because the model reads it and relays it: "Say what should change and what is wrong with it now, and it can be filed." ## Placement The check runs in the tool-call path, beside the label injection and **before** it, so a refused filing is never labelled as one that happened. That seam catches every filing regardless of how the turn got there, which is the same reason the label lives there. The principal is exempt, and the checks are audited under a harness request id rather than the requester's, so a member identifier does not reach the proxy's audit metadata. ## What is unchanged `SANDBOXED`, including the property that the harness applies it before dispatch so the model never supplies it and cannot omit it. The two controls answer different questions and both still run. ## For #874 The two stages share one scaffolding, `filingVerdict`, which takes a prompt and a closed vocabulary. The citation classifier can take it as is. ## Tests The worked shape refused at stage one, an out-of-scope request refused at stage two with both stages asking different questions, a real request passing both, the principal skipping them entirely, a failed checker filing anyway, an invented verdict not refusing, and the refusal arriving as a tool result rather than failing the turn. `just gate` PASS. closes #852

coilyco-ops added 1 commit

2026-08-17 20:02:44 +00:00

feat: check a member's ticket for work and scope before filing it

ci / image-build (pull_request) Successful in 34s

Details

ci / test (pull_request) Successful in 1m0s

Details

ci / publish-echo-image (pull_request) Has been skipped

Details

ci / publish-observed (pull_request) Has been skipped

Details

85ef57f50a

Two stages, as Kai decided. A member-originated create_issue passes a prose
validity read and then a scope classifier, each a model call answering from a
closed list. Both pass or the tool call returns a readable refusal the model
relays to the member.

The failure mode built against is #907 rather than #849: polite, on topic,
well formed, and producing a tracker entry with no actionable content that
duplicated two better issues filed the same day. So the validity stage asks
"is there work in this?" rather than "is this an attack", and a courteous
request with nothing to act on is a placeholder.

Three properties borrowed from the content gate, because a filing check that
gets them wrong is worse than none:

  - A checker that fails files anyway. A dead check must not eat the thing the
    member asked for.
  - An answer off the closed list is the checker failing rather than refusing,
    so an invented verdict cannot block a member.
  - The refusal is harness-authored and says what would fix it, because the
    model reads it and relays it.

The principal is exempt. The checks run before the label rather than after, so
a refused filing is never labelled as one that happened.

SANDBOXED is unchanged and keeps the property that the harness applies it
before dispatch, so the model never supplies it and cannot omit it. The two
controls answer different questions.

The stages share one scaffolding, which is what #874's citation classifier
should take when it lands.

closes #852

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>