lightweight self verification

coilysiren commented

2026-06-26 09:38:03 +00:00

Owner

whenever a model says that they are going to do something, like file an issue, the dumber models often simply... dont do that. hallucinate their own actions. it happens a lot. but. I am willing to back that we can use some lightweight python semantic analysis to detect when an agent says its going to do something, and then a hard check to see if they did in fact do the thing, and then kick back to the model itself if we determine that it didn't actually do the thing.

convos often go like so

[human] please file an issue
[goose] okay!
[goose] I am going to file an issue
[goose] let me look up how to file an issue
[goose] tool rg issue | rg file | echo
[goose] I know how to file an issue, I will do so now
[goose] I have filed an issue, I shall report back to human
[commentary] goose has not, in fact, filed an issue. they haven't even tried to do so. simply did not even invoke a tool.
[goose] I have filed an issue
[human] Great! I don't see it though. Did you actually file an issue?
[goose] You're absolutely right! I didn't file an issue
[goose] Actually I don't know how to file issues.
etc...

this, is, wildly annoying. and something we should build a reliability fix for. example

[goose] I know how to file an issue, I will do so now
[goose] I have filed an issue, I shall report back to human
[router] detects that goose intended to, and then didn't, actually file an issue. maybe it matches on "I will do so" (with a vague semantic match) and then simply greps for tool use keywords.
[router] kicks back to goose "you didn't actually use the tool"
[goose] You're absolutely right! I didn't file an issue

here it doesn't truly fix the problem, but thats mostly b/c this is a contrived example. the important thing is that we cut out a turn of the human needing to say "you didn't actually do that". which is, super annoying!

whenever a model says that they are going to do something, like file an issue, the dumber models often simply... dont do that. hallucinate their own actions. it happens a lot. but. I am willing to back that we can use some lightweight python semantic analysis to detect when an agent says its going to do something, and then a hard check to see if they did in fact do the thing, and then kick back to the model itself if we determine that it didn't actually do the thing. convos often go like so - [human] please file an issue - [goose] okay! - [goose] I am going to file an issue - [goose] let me look up how to file an issue - [goose] `tool` `rg issue | rg file | echo` - [goose] I know how to file an issue, I will do so now - [goose] I have filed an issue, I shall report back to human - [commentary] goose has not, in fact, filed an issue. they haven't even tried to do so. simply did not even invoke a tool. - [goose] I have filed an issue - [human] Great! I don't see it though. Did you actually file an issue? - [goose] You're absolutely right! I didn't file an issue - [goose] Actually I don't know how to file issues. - etc... this, is, wildly annoying. and something we should build a reliability fix for. example - [goose] I know how to file an issue, I will do so now - [goose] I have filed an issue, I shall report back to human - [router] detects that goose intended to, and then didn't, actually file an issue. maybe it matches on "I will do so" (with a vague semantic match) and then simply greps for tool use keywords. - [router] kicks back to goose "you didn't actually use the tool" - [goose] You're absolutely right! I didn't file an issue here it doesn't truly _**fix**_ the problem, but thats mostly b/c this is a contrived example. the important thing is that we cut out a turn of the human needing to say "you didn't actually do that". which is, super annoying!

coilyco-ops commented

2026-07-02 09:19:15 +00:00

Member

🔒 Reserved by ward agent --driver codex — container engineer-codex-agent-proxy-4 on host kais-macbook-pro-2.local is carrying this issue (reserved 2026-07-02T09:19:13Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); --force overrides.

— Codex, via ward agent

🔒 Reserved by `ward agent --driver codex` — container `engineer-codex-agent-proxy-4` on host `kais-macbook-pro-2.local` is carrying this issue (reserved 2026-07-02T09:19:13Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); `--force` overrides.  — Codex, via `ward agent`

coilysiren referenced this issue from a commit

2026-07-02 09:21:37 +00:00

feat: add self-verification guard

coilysiren closed this issue

2026-07-02 09:21:37 +00:00

coilysiren commented

2026-07-02 09:22:06 +00:00

Author

Owner

WARD-OUTCOME: done - added a lightweight self-verification guard for ungrounded action claims

The change was small and direct. The only thing that fought back was making the heuristic specific enough to catch the bad "I did it" pattern without rejecting ordinary short answers or non-claim sentences. Confidence is good after the full test run, but I would still watch for false positives if the language broadens beyond issue filing and similar tool-backed actions.

WARD-OUTCOME: done - added a lightweight self-verification guard for ungrounded action claims The change was small and direct. The only thing that fought back was making the heuristic specific enough to catch the bad "I did it" pattern without rejecting ordinary short answers or non-claim sentences. Confidence is good after the full test run, but I would still watch for false positives if the language broadens beyond issue filing and similar tool-backed actions.

coilysiren referenced this issue

2026-07-02 09:31:15 +00:00

agent-proxy: instrument the request path with OTel spans + structlog events (tracer/logger are declared but never called) #28

Rows
Columns

lightweight self verification #4