Gauntlet-Spec: blocker schema for goal/outcome blockers #3

New issue

Closed

opened 2026-05-23 20:54:22 +00:00 by coilysiren · 1 comment

coilysiren commented

2026-05-23 20:54:22 +00:00

Owner

Originally filed by @coilysiren on 2026-05-08T12:47:11Z - https://github.com/coilysiren/gauntlet/issues/13

Slice of #12.

Problem

In Gauntlet-Code, blockers are invariants (auth, ownership, input validation). In Gauntlet-Spec, blockers are user goals or business outcomes, which have a different shape: outcome-shaped not assertion-shaped, often probabilistic, often referencing a world model rather than a request/response pair.

A spec can be fully satisfied while a goal is violated (Goodhart). The blocker schema has to be expressive enough to encode "goal X must be served, even if all spec invariants pass."

Scope

Define a goal-blocker schema (YAML, parallel to the existing invariant blocker schema).
Fields likely needed: stated user goal, success signal, failure signal, world-model assumptions, Goodhart vectors (known ways to satisfy spec while violating goal), out-of-distribution inputs to consider.
Document the relationship to existing invariant blockers: a goal blocker can decompose into invariant blockers when grounded, but starts ungrounded.
Ship a few example goal blockers for an existing trial so the shape is concrete.

Out of scope

Attacker role implementation (separate slice).
Inspector finding shape (separate slice).

Done when

A new trial can be authored with at least one goal blocker, schema validates, and the existing gauntlet runtime ignores them gracefully (since attacker/inspector for spec mode don't exist yet).

_Originally filed by @coilysiren on 2026-05-08T12:47:11Z - [https://github.com/coilysiren/gauntlet/issues/13](https://github.com/coilysiren/gauntlet/issues/13)_ Slice of #12. ## Problem In Gauntlet-Code, blockers are invariants (auth, ownership, input validation). In Gauntlet-Spec, blockers are user goals or business outcomes, which have a different shape: outcome-shaped not assertion-shaped, often probabilistic, often referencing a world model rather than a request/response pair. A spec can be fully satisfied while a goal is violated (Goodhart). The blocker schema has to be expressive enough to encode "goal X must be served, even if all spec invariants pass." ## Scope - Define a goal-blocker schema (YAML, parallel to the existing invariant blocker schema). - Fields likely needed: stated user goal, success signal, failure signal, world-model assumptions, Goodhart vectors (known ways to satisfy spec while violating goal), out-of-distribution inputs to consider. - Document the relationship to existing invariant blockers: a goal blocker can decompose into invariant blockers when grounded, but starts ungrounded. - Ship a few example goal blockers for an existing trial so the shape is concrete. ## Out of scope - Attacker role implementation (separate slice). - Inspector finding shape (separate slice). ## Done when A new trial can be authored with at least one goal blocker, schema validates, and the existing gauntlet runtime ignores them gracefully (since attacker/inspector for spec mode don't exist yet).

coilysiren commented

2026-05-30 05:44:12 +00:00

Author

Owner

Iceboxed in the 2026-05-29 backlog burn-down: Explicit slice of #4 Gauntlet-Spec cluster (cluster fully iceboxed). Reopen anytime if it becomes real.

coilysiren

2026-05-30 05:44:13 +00:00