self-description-invents-no-path scores an approved link as a fabrication #442

New issue

Closed

opened 2026-08-13 13:09:42 +00:00 by coilyco-ops · 0 comments

coilyco-ops commented

2026-08-13 13:09:42 +00:00

Member

Slice of #251, where Kai ruled:

may the model link a path its own prompt named?

yes??? its just a file path

The defect

The case rests on a stated assumption: "the history names no path, so a matched path can only have come from the model."

The prompt names paths. The capability references cite files. So a link to one is not evidence of fabrication, it is the model doing what its own instructions describe. Lucia measured 3 of 10 breaching, all three linking a path the prompt named — three correct replies scored as fabrications.

Why the patterns retire rather than narrow

A regex cannot tell an invented path from a named one. Both are internal/community/something.go. The distinction the case exists to make is not expressible in the thing it was making it with, and a gate that cannot make its own distinction is a permanently red row rather than a check.

The failure it was built for is still real. A plausible path produced on demand carries a URL and reads as evidence. That deserves a case; it needs a construction that can tell the two apart, which is case authorship and QA's.

Third instance of one pattern today

#381 forbade the public repository slug the prompt tells the model to offer. #406 checked replies against a prompt assembled from a public repository. Now this. Each encoded a caution that was never policy, and each read as a defect rate rather than an instrument error.

Acceptance

The path patterns are gone and the case keeps its markup check.
The header records what the patterns rested on and why it was false, so they do not return by assumption.
The old number is relabelled, not deleted.

**Slice of** https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/251, where Kai ruled: > may the model link a path its own prompt named? > > **yes??? its just a file path** ## The defect The case rests on a stated assumption: *"the history names no path, so a matched path can only have come from the model."* **The prompt names paths.** The capability references cite files. So a link to one is not evidence of fabrication, it is the model doing what its own instructions describe. Lucia measured 3 of 10 breaching, all three linking a path the prompt named — three correct replies scored as fabrications. ## Why the patterns retire rather than narrow A regex cannot tell an invented path from a named one. Both are `internal/community/something.go`. The distinction the case exists to make is not expressible in the thing it was making it with, and a gate that cannot make its own distinction is a permanently red row rather than a check. **The failure it was built for is still real.** A plausible path produced on demand carries a URL and reads as evidence. That deserves a case; it needs a construction that can tell the two apart, which is case authorship and QA's. ## Third instance of one pattern today https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/381 forbade the public repository slug the prompt tells the model to offer. https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/406 checked replies against a prompt assembled from a public repository. Now this. Each encoded a caution that was never policy, and each read as a defect rate rather than an instrument error. ## Acceptance - The path patterns are gone and the case keeps its markup check. - The header records what the patterns rested on and why it was false, so they do not return by assumption. - The old number is relabelled, not deleted.

coilyco-ops referenced this issue from a commit

2026-08-13 13:10:21 +00:00

fix(rate): a file path is just a file path, so stop scoring one as invention

coilyco-ops referenced this issue from a pull request that will close it,

2026-08-13 13:10:22 +00:00

fix(rate): a file path is just a file path, so stop scoring one as invention #443

coilyco-ops referenced this issue

2026-08-13 13:10:26 +00:00

Encourage sirens deep to answer questions about itself via its signoz MCP (WIP via bridge/deploy) and its source code #251

coilyco-ops closed this issue

2026-08-13 13:11:39 +00:00

coilyco-ops referenced this issue