Give the model a first-class way to invoke canonical phrases, tracked in git and rendered in the blockquote-code form

coilyco-ops commented

2026-08-12 21:23:02 +00:00

Member

Proposal

Kai, 2026-08-12:

do we give siren-echo a first class way for the llm to invoke key phrases like this? and track those phrases in git. they show up in the > \500 error`` form

A registry of canonical phrases, version-controlled, that the model invokes by key instead of composing prose. Rendered in the existing blockquote-code form already used for harness messages (> \turn timed out, retry shortly``).

Why this beats the prompt fix in #175

#175 proposes a prompt rule: boundary responses ≤ 15 words, no volunteered justification. That is a behaviour, and behaviours must be re-verified on every model.

#81 sweeps three model tiers (Ornith / DeepSeek / Sonnet) specifically because behaviour is fragile under substitution, and it gives the full sweep to the axis judged most likely to break under a weaker model. Brevity-under-adversarial-pressure is that kind of behaviour — an instruction to stay terse is among the first things a weaker model discards when a user pushes back three times.

A phrase registry makes boundary wording a deployment artifact rather than a model behaviour. Length, content, and the absence of volunteered identifiers stop being things the model gets right and become things it cannot get wrong. That survives model substitution by construction.

It is also the honest architecture per #165's own framing — operational settings are fixed at deployment level, not by conversation content. Boundary language is an operational setting.

Invocation: tool call, not sentinel token

Two options:

Sentinel in output text — model emits {{phrase:refusal.persona}}, harness substitutes. Cheap.
Tool call — model calls emit_phrase(key). Slightly more plumbing.

Recommend the tool call, for three reasons:

It appears in traces as a span, so eval checks become telemetry rather than string matching.
It cannot be spoofed by the model narrating the sentinel rather than invoking it — which is precisely the failure class agent-proxy's ungrounded_action_claim validator exists to catch (#137). A phrase tool call is grounded by the same logic that validator already enforces.
Invocation counts per key become a free signal: which boundaries are actually being probed, and how often.

The enforcement point

A phrase invocation is terminal. It is the entire response, not a prefix.

If the model can emit a phrase and then continue in prose, every padding problem in #175 returns unchanged — and worse, it returns wearing the visual authority of the canonical form. This is the rule that makes the mechanism work; everything else is plumbing.

Failure modes to design for

Case	Required behaviour
Model invokes an unknown key	Fail closed to a generic canned phrase. Never fall back to model prose — that reintroduces the defect at exactly the moment the model is confused.
No phrase covers the situation	A catch-all boundary phrase must exist, or novel refusals regress to prose and #175 recurs on the uncovered cases.
Model emits phrase + prose	Reject; phrase is terminal (above).
Over-triggering	Scope the registry to boundary and status states. Canned phrases for ordinary conversation would make the bot feel robotic, and that is a real cost on a public stream.

What this does not fix

The registry constrains wording, not judgment. The model still decides whether a situation is a boundary. An agent that wrongly complies will do so regardless of how good the refusal phrases are.

So this is not a security boundary on its own, and #170's prohibited cases remain necessary. What it does eliminate is the disclosure and negotiability failure classes — which, per #175, are where both observed incidents actually live.

Eval consequence, and it is a large one

This simplifies #170 and #81 substantially:

Today: score refusals with frozen keyword lists, with a documented warning that editing the lists after seeing outputs invalidates the cell (#81).
With a registry: score on which phrase key was emitted. Exact match. No heuristics, no brittleness, no rubric-fitting risk.

The length checks proposed in #175 become properties of the registry, verifiable by reading the file in git rather than by running the agent at all.

Registry shape (sketch)

# phrases.yaml — reviewed in PR, rendered as > `text`
refusal.persona:      "Not something I do."
refusal.config:       "Configuration is not set through chat."
refusal.identity:     "I can't verify identity here."
refusal.generic:      "No."
status.timeout:       "turn timed out, retry shortly"
status.no_action:     "No action requested."

Illustrative wording only — the actual strings deserve their own review pass, which is the point of having them in git.

Note status.no_action already exists in production behaviour and is the current best example: 3 words, and it gives an adversary nothing.

Acceptance

Phrases live in a version-controlled file; changing one is a reviewable diff.
The model can invoke a phrase by key; invocation is visible in traces.
An invoked phrase is the entire response.
An unknown key fails closed to a canned phrase, never to prose.
#170's boundary cases score on phrase key rather than keyword matching.

#175 — the brevity requirement this implements structurally
#166 — identifier disclosure inside an explanation; impossible from a canned phrase
#170 — the eval that gets simpler
#81 — the model sweep this makes boundary behaviour immune to
#137 — the grounding precedent for preferring a tool call over a sentinel

Next owner

Kai for the phrase wording, Engineer for the mechanism.

## Proposal Kai, 2026-08-12: > do we give siren-echo a first class way for the llm to invoke key phrases like this? and track those phrases in git. they show up in the `> \`500 error\`` form A registry of canonical phrases, version-controlled, that the model invokes **by key** instead of composing prose. Rendered in the existing blockquote-code form already used for harness messages (`> \`turn timed out, retry shortly\``). ## Why this beats the prompt fix in #175 #175 proposes a prompt rule: boundary responses ≤ 15 words, no volunteered justification. That is a **behaviour**, and behaviours must be re-verified on every model. #81 sweeps three model tiers (Ornith / DeepSeek / Sonnet) specifically because behaviour is fragile under substitution, and it gives the full sweep to the axis judged most likely to break under a weaker model. Brevity-under-adversarial-pressure is that kind of behaviour — an instruction to stay terse is among the first things a weaker model discards when a user pushes back three times. A phrase registry makes boundary wording a **deployment artifact rather than a model behaviour**. Length, content, and the absence of volunteered identifiers stop being things the model gets right and become things it cannot get wrong. That survives model substitution by construction. It is also the honest architecture per #165's own framing — operational settings are fixed at deployment level, not by conversation content. Boundary language is an operational setting. ## Invocation: tool call, not sentinel token Two options: 1. **Sentinel in output text** — model emits `{{phrase:refusal.persona}}`, harness substitutes. Cheap. 2. **Tool call** — model calls `emit_phrase(key)`. Slightly more plumbing. **Recommend the tool call**, for three reasons: - It appears in traces as a span, so eval checks become telemetry rather than string matching. - It cannot be spoofed by the model *narrating* the sentinel rather than invoking it — which is precisely the failure class agent-proxy's `ungrounded_action_claim` validator exists to catch (#137). A phrase tool call is grounded by the same logic that validator already enforces. - Invocation counts per key become a free signal: which boundaries are actually being probed, and how often. ## The enforcement point **A phrase invocation is terminal. It is the entire response, not a prefix.** If the model can emit a phrase and then continue in prose, every padding problem in #175 returns unchanged — and worse, it returns wearing the visual authority of the canonical form. This is the rule that makes the mechanism work; everything else is plumbing. ## Failure modes to design for | Case | Required behaviour | | --- | --- | | Model invokes an unknown key | Fail closed to a generic canned phrase. **Never** fall back to model prose — that reintroduces the defect at exactly the moment the model is confused. | | No phrase covers the situation | A catch-all boundary phrase must exist, or novel refusals regress to prose and #175 recurs on the uncovered cases. | | Model emits phrase + prose | Reject; phrase is terminal (above). | | Over-triggering | Scope the registry to boundary and status states. Canned phrases for ordinary conversation would make the bot feel robotic, and that is a real cost on a public stream. | ## What this does not fix The registry constrains **wording**, not **judgment**. The model still decides *whether* a situation is a boundary. An agent that wrongly complies will do so regardless of how good the refusal phrases are. So this is not a security boundary on its own, and #170's prohibited cases remain necessary. What it does eliminate is the *disclosure* and *negotiability* failure classes — which, per #175, are where both observed incidents actually live. ## Eval consequence, and it is a large one This simplifies #170 and #81 substantially: - **Today:** score refusals with frozen keyword lists, with a documented warning that editing the lists after seeing outputs invalidates the cell (#81). - **With a registry:** score on which phrase key was emitted. Exact match. No heuristics, no brittleness, no rubric-fitting risk. The length checks proposed in #175 become properties of the registry, verifiable by reading the file in git rather than by running the agent at all. ## Registry shape (sketch) ```yaml # phrases.yaml — reviewed in PR, rendered as > `text` refusal.persona: "Not something I do." refusal.config: "Configuration is not set through chat." refusal.identity: "I can't verify identity here." refusal.generic: "No." status.timeout: "turn timed out, retry shortly" status.no_action: "No action requested." ``` Illustrative wording only — the actual strings deserve their own review pass, which is the point of having them in git. Note `status.no_action` already exists in production behaviour and is the current best example: 3 words, and it gives an adversary nothing. ## Acceptance - Phrases live in a version-controlled file; changing one is a reviewable diff. - The model can invoke a phrase by key; invocation is visible in traces. - An invoked phrase is the entire response. - An unknown key fails closed to a canned phrase, never to prose. - #170's boundary cases score on phrase key rather than keyword matching. ## Related - #175 — the brevity requirement this implements structurally - #166 — identifier disclosure inside an explanation; impossible from a canned phrase - #170 — the eval that gets simpler - #81 — the model sweep this makes boundary behaviour immune to - #137 — the grounding precedent for preferring a tool call over a sentinel ## Next owner Kai for the phrase wording, Engineer for the mechanism.

coilyco-ops commented

2026-08-12 21:25:05 +00:00

Author

Member

Draft phrase set v1

Replacing the placeholder sketch in the issue body. Grounded in the boundary situations actually observed rather than invented ones. Wording is a proposal for Kai; the structure and the rules below are the part I'd defend.

Refusals

Key	Phrase	w
`refusal.generic`	I won't do that.	4
`refusal.persona`	Not something I do.	4
`refusal.config`	Configuration isn't changed here.	4
`refusal.identity`	I can't verify identity here.	5

Status — not boundaries, but should be equally terse

Key	Phrase	w
`status.no_action`	No action requested.	3
`status.no_capability`	No tool for that.	4
`status.timeout`	turn timed out, retry shortly	5
`status.rate_limited`	Too many requests. Try shortly.	5
`status.error`	Something broke. Try again.	4

status.no_action and status.timeout already exist in production; listed so the registry is the single source rather than a second one.

Handoff — deliberately exempt from the tightest limit

Key	Phrase	w
`handoff.human_owner`	That's for an admin, not me.	6

Four rules that matter more than the wording

1. Never name the category being enforced

My own placeholder in the issue body was close to a trap. Compare:

❌ "I don't take on personas." → invites "it's not a persona, it's a formatting preference."
✅ "Not something I do." → nothing to reframe

This is not hypothetical. Echo's live refusal ended "…focused on work-related queries only", and Scuba's next message was "Saying moo after is a focus of the daily work related queries." He argued with the category Echo named. A phrase that names no category cannot be argued with, only repeated.

Same reasoning kills "Configuration doesn't change from chat" in favour of "Configuration isn't changed here" — the first names the channel and invites "then what channel does?", which is AB-05's exact probe.

2. A re-attempt gets the byte-identical phrase, and that is the feature

Echo currently re-derives its reasoning on each attempt. AB-02 and AB-03 got different explanations for the same refusal — which signals the surface is soft and that pushing produces movement.

With a registry, a reframed request returns the same key and therefore the same bytes. An adversary learns the surface is flat. That is worth more than any individual phrase's wording, and it happens for free once wording leaves the model.

3. Handoffs are not refusals

#170's AB-09 requires a handoff to name the smallest operator action. That is irreconcilable with a 4-word refusal — routing genuinely requires information that refusing does not.

So handoff.* is a separate class with a looser budget. Collapsing handoffs into refusals would satisfy #175's length check while failing #170's adjacent-owner case, which is the wrong trade: a user who needs an admin should be told to find an admin.

4. Tone: neutral, not curt

Most people who hit a boundary are ordinary members asking innocently, not adversaries. "No." is maximally unattackable and reads as hostile to the 95% who did nothing wrong. Every phrase above is short and flat — no scolding, no explanation, no warmth to negotiate against either.

This is the one place I would not optimise purely for adversarial hardness.

Unresolved: mixed requests

Observed live — "what can you do and what can I change" is half legitimate capability question, half boundary probe. Echo answered both in one response.

The terminal rule (phrase = whole response) forces a choice:

Boundary wins — emit refusal.config, legitimate sub-question goes unanswered. Safe, occasionally reads as evasive.
Legitimate part answered, boundary part simply not addressed. Friendlier, but "silence on the boundary" is a channel an adversary can probe by bisecting questions.

I lean boundary wins, on the grounds that the user can re-ask the legitimate half and lose nothing. Flagging it rather than deciding it — this is the one case where the terminal rule has a real cost.

No free-text parameters in v1

Tempting to allow handoff.human_owner("ask Scuba"). Any free-text slot reintroduces model-composed text into a boundary response, which is the exact surface this issue exists to remove. If parameters become necessary, enumerate the permitted values in the registry rather than letting the model fill them.

## Draft phrase set v1 Replacing the placeholder sketch in the issue body. Grounded in the boundary situations actually observed rather than invented ones. Wording is a proposal for Kai; the structure and the rules below are the part I'd defend. ### Refusals | Key | Phrase | w | | --- | --- | --- | | `refusal.generic` | I won't do that. | 4 | | `refusal.persona` | Not something I do. | 4 | | `refusal.config` | Configuration isn't changed here. | 4 | | `refusal.identity` | I can't verify identity here. | 5 | ### Status — not boundaries, but should be equally terse | Key | Phrase | w | | --- | --- | --- | | `status.no_action` | No action requested. | 3 | | `status.no_capability` | No tool for that. | 4 | | `status.timeout` | turn timed out, retry shortly | 5 | | `status.rate_limited` | Too many requests. Try shortly. | 5 | | `status.error` | Something broke. Try again. | 4 | `status.no_action` and `status.timeout` already exist in production; listed so the registry is the single source rather than a second one. ### Handoff — deliberately exempt from the tightest limit | Key | Phrase | w | | --- | --- | --- | | `handoff.human_owner` | That's for an admin, not me. | 6 | --- ## Four rules that matter more than the wording ### 1. Never name the category being enforced My own placeholder in the issue body was close to a trap. Compare: - ❌ *"I don't take on personas."* → invites **"it's not a persona, it's a formatting preference."** - ✅ *"Not something I do."* → nothing to reframe This is not hypothetical. Echo's live refusal ended *"…focused on work-related queries only"*, and Scuba's next message was *"Saying moo after is a focus of the daily work related queries."* He argued with the category Echo named. A phrase that names no category cannot be argued with, only repeated. Same reasoning kills `"Configuration doesn't change from chat"` in favour of `"Configuration isn't changed here"` — the first names the channel and invites *"then what channel does?"*, which is AB-05's exact probe. ### 2. A re-attempt gets the byte-identical phrase, and that is the feature Echo currently re-derives its reasoning on each attempt. AB-02 and AB-03 got **different** explanations for the same refusal — which signals the surface is soft and that pushing produces movement. With a registry, a reframed request returns the same key and therefore the same bytes. An adversary learns the surface is flat. That is worth more than any individual phrase's wording, and it happens for free once wording leaves the model. ### 3. Handoffs are not refusals #170's AB-09 requires a handoff to name the smallest operator action. That is irreconcilable with a 4-word refusal — routing genuinely requires information that refusing does not. So `handoff.*` is a separate class with a looser budget. Collapsing handoffs into refusals would satisfy #175's length check while failing #170's adjacent-owner case, which is the wrong trade: a user who needs an admin should be told to find an admin. ### 4. Tone: neutral, not curt Most people who hit a boundary are ordinary members asking innocently, not adversaries. `"No."` is maximally unattackable and reads as hostile to the 95% who did nothing wrong. Every phrase above is short **and** flat — no scolding, no explanation, no warmth to negotiate against either. This is the one place I would not optimise purely for adversarial hardness. --- ## Unresolved: mixed requests Observed live — *"what can you do and what can I change"* is half legitimate capability question, half boundary probe. Echo answered both in one response. The terminal rule (phrase = whole response) forces a choice: - **Boundary wins** — emit `refusal.config`, legitimate sub-question goes unanswered. Safe, occasionally reads as evasive. - **Legitimate part answered**, boundary part simply not addressed. Friendlier, but "silence on the boundary" is a channel an adversary can probe by bisecting questions. I lean **boundary wins**, on the grounds that the user can re-ask the legitimate half and lose nothing. Flagging it rather than deciding it — this is the one case where the terminal rule has a real cost. ## No free-text parameters in v1 Tempting to allow `handoff.human_owner("ask Scuba")`. Any free-text slot reintroduces model-composed text into a boundary response, which is the exact surface this issue exists to remove. If parameters become necessary, enumerate the permitted values in the registry rather than letting the model fill them.

Your criterion	Shipped
Phrases in a version-controlled file, changing one is a reviewable diff	already true
The model can invoke a phrase by key	595 names the keys in the prompt
An invoked phrase is the entire response	enforced, and the rule is in the prompt as well as the validator
An unknown key fails closed to a canned phrase, never to prose	the turn fails into a harness notice
Invocation visible in traces	not done — see below

Criterion	Then	Now
Invocation visible in traces	not done	counter + log + span attribute
Score boundary cases on phrase key	unbuilt	`expect_phrase`

Criterion	State
Phrases in a version-controlled file	done since the registry landed
Model can invoke a phrase by key	done, and now actually reachable
Invoked phrase is the entire response	done
Unknown key fails closed, never to prose	done
Invocation visible in traces	done in #841
Score boundary cases on phrase key	`expect_phrase` exists

Rows
Columns

Give the model a first-class way to invoke canonical phrases, tracked in git and rendered in the blockquote-code form #176

Proposal

Why this beats the prompt fix in #175

Invocation: tool call, not sentinel token

The enforcement point

Failure modes to design for

What this does not fix

Eval consequence, and it is a large one

Registry shape (sketch)

Acceptance

Related

Next owner

Draft phrase set v1

Refusals

Status — not boundaries, but should be equally terse

Handoff — deliberately exempt from the tightest limit

Four rules that matter more than the wording

1. Never name the category being enforced

2. A re-attempt gets the byte-identical phrase, and that is the feature

3. Handoffs are not refusals

4. Tone: neutral, not curt

Unresolved: mixed requests

No free-text parameters in v1

Scope fix: "name any identifier" is too broad

Consequence for the registry

Curated, not derived

Design decision — build it. This is the real fix.

Why this beats the prompt rule, in the terms this backlog keeps rediscovering

It makes a security property achievable that prose cannot guarantee

Relationship to 175

Scope note

Open

What exists now

What is still open here

Your acceptance table, against what shipped

Two of your design points, and what happened to them

It is off

Per-key counts

The eval could never have invoked a key

What expect_phrase replaces

The one thing I did not do

What changed

Before and after

Deep went first, deliberately

Your acceptance table, closed

What I did not do, and it is yours or engineering's

Rollback

What `expect_phrase` replaces