coilyco-flight-deck/agentic-os

Watch

Nothing validates .agents/roles.kdl against the composed catalogue, which is why a deleted role block and three orphaned methods both went unnoticed #1076

New issue

Closed

opened 2026-08-15 23:21:11 +00:00 by coilyco-ops · 1 comment

coilyco-ops commented

2026-08-15 23:21:11 +00:00

Member

Filed by Darren (director seat), 2026-08-15. Two bugs landed within an hour of each other today, both in the same blind spot, and neither was found by a validator. #1071 found one by reading a file. #1072 found the other while fixing the first.

The gap, verified in source

check-composed-skills is the only hook over .agents/composed/. It is 90 lines in agentic_os/pre_commit/check_composed_skills.py, its one worker is layout_problems, and it never mentions roles.kdl, composed-skill, or selection at all. Its own description says what it covers:

Validate role-composed COMPOSED.md sources against the ordinary skill taxonomy while preventing native discovery and name collisions.

That is layout and naming. Not selection.

A repo-wide grep for roles.kdl returns docs, plus one test - tests/test_context_budget_role.py, which writes its own fixture roles.kdl and never reads the real one.

So no check ties the catalogue to the role graph in either direction.

What that cost, twice, today

#1071 - the exec block was parked in a block comment and then deleted by 48c48414 when the code-comments hook stopped exempting roles.kdl. A role with no block selects nothing, so every Executive Strategist bundle composed since 2026-08-08 carried zero AOS methods. Found by reading the file.
#1072 - three composed sources were orphaned when Community stopped being a deployed role. docs/role-skill-coverage-audit.md went on asserting that everything except the two sales methods was claimed, which had quietly become false.

Worth naming the shape: a hook change silently deleted a config, and nothing downstream noticed. That is the same failure family as a label rename emptying a string matcher - one place changes, another place quietly stops selecting, and the symptom is absence rather than an error.

Two assertions, and they catch different things

One: every composed source is claimed by at least one role. Catches #1072. Roughly the check I ran by hand to verify that PR:

sel = parse_composed_skill_patterns(".agents/roles.kdl")   # role -> [glob, ...]
sources = {p.name for p in Path(".agents/composed").iterdir() if p.is_dir()}
unclaimed = [s for s in sources
             if not any(fnmatch(s, pat) for pats in sel.values() for pat in pats)]

Two: every deployed role has a block, and every selector resolves. Catches #1071, and the first assertion does not - a missing role block leaves no unclaimed source when other roles happen to cover the same globs.

Run against main right now: unclaimed is exactly the two sales methods, and no selector matches zero sources. So both assertions pass today and the check would start green.

The design content is the exemption, not the check

A naive fail-on-unclaimed fails immediately on tooling-sales-discovery and tooling-sales-negotiation-architecture, which are deliberately unclaimed - AOS deploys no sales role and docs/role-skill-coverage-audit.md says so on purpose. So the check needs a declared exemption rather than a bare assertion, and the exemption belongs next to the data rather than in the validator.

That matters more than it looks. An exemption list is a place where "we know this is unclaimed and here is why" gets written down once, which is exactly the record that was missing when three sources went orphaned and a doc kept claiming otherwise.

Where deployed roles come from, offline

Pre-commit runs offline, so agent-compose catalog roles is not available to the hook. .agents/harness-launch-profiles.yaml carries the same eight keys in-repo and is the natural source. It is also the file that disagreed with roles.kdl in #1071 while nothing compared them, which makes it the right thing to compare against.

Do

Extend check-composed-skills, or add a sibling hook, with the two assertions.
Declare the sales exemption where the reason lives with it.
Compare the role set in roles.kdl against .agents/harness-launch-profiles.yaml and fail on a role present in one and absent from the other.

Acceptance

Deleting a role block from roles.kdl fails a hook.
Adding a composed source that no role selects fails a hook, unless it is a declared exemption.
A selector matching zero sources fails a hook.
The check runs offline and passes on main unchanged.

Found while reviewing #1071 and fixing #1072. Authoring belongs here per the authoring-versus-rollout split; no fleet rollout is involved.

**Filed by Darren (director seat), 2026-08-15.** Two bugs landed within an hour of each other today, both in the same blind spot, and neither was found by a validator. #1071 found one by reading a file. #1072 found the other while fixing the first. ## The gap, verified in source `check-composed-skills` is the only hook over `.agents/composed/`. It is 90 lines in `agentic_os/pre_commit/check_composed_skills.py`, its one worker is `layout_problems`, and it never mentions `roles.kdl`, `composed-skill`, or selection at all. Its own description says what it covers: > Validate role-composed COMPOSED.md sources against the ordinary skill taxonomy while preventing native discovery and name collisions. That is layout and naming. Not selection. A repo-wide grep for `roles.kdl` returns docs, plus one test - `tests/test_context_budget_role.py`, which writes its **own fixture** `roles.kdl` and never reads the real one. **So no check ties the catalogue to the role graph in either direction.** ## What that cost, twice, today * **#1071** - the `exec` block was parked in a block comment and then deleted by `48c48414` when the code-comments hook stopped exempting `roles.kdl`. A role with no block selects nothing, so **every Executive Strategist bundle composed since 2026-08-08 carried zero AOS methods.** Found by reading the file. * **#1072** - three composed sources were orphaned when Community stopped being a deployed role. `docs/role-skill-coverage-audit.md` went on asserting that everything except the two sales methods was claimed, which had quietly become false. Worth naming the shape: **a hook change silently deleted a config**, and nothing downstream noticed. That is the same failure family as a label rename emptying a string matcher - one place changes, another place quietly stops selecting, and the symptom is absence rather than an error. ## Two assertions, and they catch different things **One: every composed source is claimed by at least one role.** Catches #1072. Roughly the check I ran by hand to verify that PR: ```python sel = parse_composed_skill_patterns(".agents/roles.kdl") # role -> [glob, ...] sources = {p.name for p in Path(".agents/composed").iterdir() if p.is_dir()} unclaimed = [s for s in sources if not any(fnmatch(s, pat) for pats in sel.values() for pat in pats)] ``` **Two: every deployed role has a block, and every selector resolves.** Catches #1071, and the first assertion does not - a missing role block leaves no unclaimed source when other roles happen to cover the same globs. Run against `main` right now: unclaimed is exactly the two sales methods, and **no selector matches zero sources**. So both assertions pass today and the check would start green. ## The design content is the exemption, not the check A naive fail-on-unclaimed fails immediately on `tooling-sales-discovery` and `tooling-sales-negotiation-architecture`, which are deliberately unclaimed - AOS deploys no sales role and `docs/role-skill-coverage-audit.md` says so on purpose. So the check needs a declared exemption rather than a bare assertion, and the exemption belongs next to the data rather than in the validator. That matters more than it looks. An exemption list is a place where "we know this is unclaimed and here is why" gets written down once, which is exactly the record that was missing when three sources went orphaned and a doc kept claiming otherwise. ## Where deployed roles come from, offline Pre-commit runs offline, so `agent-compose catalog roles` is not available to the hook. `.agents/harness-launch-profiles.yaml` carries the same eight keys in-repo and is the natural source. It is also the file that **disagreed with `roles.kdl` in #1071** while nothing compared them, which makes it the right thing to compare against. ## Do 1. Extend `check-composed-skills`, or add a sibling hook, with the two assertions. 2. Declare the sales exemption where the reason lives with it. 3. Compare the role set in `roles.kdl` against `.agents/harness-launch-profiles.yaml` and fail on a role present in one and absent from the other. ## Acceptance * Deleting a `role` block from `roles.kdl` fails a hook. * Adding a composed source that no role selects fails a hook, unless it is a declared exemption. * A selector matching zero sources fails a hook. * The check runs offline and passes on `main` unchanged. --- Found while reviewing #1071 and fixing #1072. Authoring belongs here per the authoring-versus-rollout split; no fleet rollout is involved.

coilyco-ops added the

priority

autonomy

headless

labels

2026-08-15 23:21:35 +00:00

coilyco-ops commented

2026-08-17 02:40:19 +00:00

Author

Member

Merged into #1073. Closing. Darren (director seat), 2026-08-17, from a full triage pass of this repo.

Both issues ask for the same validator, both were filed on 2026-08-15, and both cite 48c48414 as the incident. #1073 is the lower number and is the canonical one.

Confirmed still unbuilt rather than assumed: agentic_os/pre_commit/ holds 27 checkers and none of them is a roles or composed-selection validator. check_composed_skills.py is layout and naming only, exactly as this issue establishes.

What #1073 already carries

Both failure directions, which is the substance:

Missing role block - a role agent-compose deploys can have no role block at all, composing zero methods while every hook passes. 48c48414 deleted the parked role exec block because code-comments correctly rejects commented-out config, and the Executive Strategist composed zero AOS methods from 2026-08-08 until someone noticed by hand. The AOSK side went the same way two days later in 9a117d4.
Orphaned composed source - a composed source no role selects is dead weight, and 48c48414's own body recorded tooling-advisor-* and tooling-ceo-* as unclaimed and shipped anyway, because a commit-message note is not a gate.

What this issue adds, carried across

The verification that check-composed-skills is the only hook over .agents/composed/, that it is 90 lines whose one worker is layout_problems, and that it never mentions roles.kdl, composed-skill, or selection. That is the evidence the gap is structural rather than a missing case, and it belongs in #1073.

The decision this gate will force

#976 is not a duplicate and stays open. It decides the fate of the five orphaned sources - tooling-advisor-causal-claim-audit, tooling-advisor-evidence-synthesis, tooling-ceo-developer-tool-adoption, tooling-ceo-oss-stewardship, tooling-ceo-platform-strategy - and the validator in #1073 turns that from a note into a red hook. Answer #976 before or alongside #1073, or the gate lands red on its own repo.

#1073 is priority/P2 autonomy/headless role/engineer. #976 is priority/P3 autonomy/async-consult role/director.

**Merged into #1073. Closing. Darren (director seat), 2026-08-17, from a full triage pass of this repo.** Both issues ask for the same validator, both were filed on 2026-08-15, and both cite `48c48414` as the incident. #1073 is the lower number and is the canonical one. Confirmed still unbuilt rather than assumed: `agentic_os/pre_commit/` holds 27 checkers and none of them is a roles or composed-selection validator. `check_composed_skills.py` is layout and naming only, exactly as this issue establishes. ## What #1073 already carries Both failure directions, which is the substance: * **Missing role block** - a role agent-compose deploys can have no `role` block at all, composing zero methods while every hook passes. `48c48414` deleted the parked `role exec` block because `code-comments` correctly rejects commented-out config, and the Executive Strategist composed zero AOS methods from 2026-08-08 until someone noticed by hand. The AOSK side went the same way two days later in `9a117d4`. * **Orphaned composed source** - a composed source no role selects is dead weight, and `48c48414`'s own body recorded `tooling-advisor-*` and `tooling-ceo-*` as unclaimed and shipped anyway, because a commit-message note is not a gate. ## What this issue adds, carried across The verification that `check-composed-skills` is the **only** hook over `.agents/composed/`, that it is 90 lines whose one worker is `layout_problems`, and that it never mentions `roles.kdl`, `composed-skill`, or selection. That is the evidence the gap is structural rather than a missing case, and it belongs in #1073. ## The decision this gate will force **#976** is not a duplicate and stays open. It decides the fate of the five orphaned sources - `tooling-advisor-causal-claim-audit`, `tooling-advisor-evidence-synthesis`, `tooling-ceo-developer-tool-adoption`, `tooling-ceo-oss-stewardship`, `tooling-ceo-platform-strategy` - and the validator in #1073 turns that from a note into a red hook. Answer #976 before or alongside #1073, or the gate lands red on its own repo. #1073 is `priority/P2` `autonomy/headless` `role/engineer`. #976 is `priority/P3` `autonomy/async-consult` `role/director`.

coilyco-ops closed this issue

2026-08-17 02:40:48 +00:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

coilyco-flight-deck/agentic-os#1076

No description provided.

Rows
Columns