coilyco-flight-deck/agent-compose
coilyco-flight-deck/agent-compose
Watch
2
0
Code Issues 20 Pull requests Projects Releases 161 Packages Wiki Activity Actions

Make composition decisions chatty and inspectable #8

New issue
Closed
opened 2026-07-16 08:01:08 +00:00 by coilyco-ops · 1 comment
coilyco-ops commented 2026-07-16 08:01:08 +00:00
Member
Copy link

Parent

#1

What to build

Make explainability a first-class product surface for agent-compose. Every resolution should produce a structured decision tree while choices are made, retain the deterministic evidence with the bundle, and render it for humans with progressive disclosure. The result should feel chatty and legible during interactive use without flooding unattended launch logs or model context.

Human experience

An interactive compose shows a bounded summary: resolved profile, target repos, cache result, changed outputs, selected source and skill counts, delivery mode, and warnings. A human can then expand the exact branch she cares about instead of paging through undifferentiated debug logs.

The detailed tree should answer:

  • Which person source, agent, role, personality, model class, privacy scope, and repo facts were used?
  • Which policy rules were evaluated, in what order, and from which source?
  • Which source, capability, and skill candidates were considered?
  • Why was each candidate included, excluded, shadowed, or replaced?
  • Which fallback fired and what preferred input was unavailable?
  • Why was content mounted natively, compiled into instructions, or omitted?
  • Which decision changed between two bundles?

Acceptance criteria

  • agent-compose describe renders the stored decision tree in stable, human-scannable sections.
  • agent-compose describe --why <source-or-skill> follows one item from repo facts through policy to its final delivery or rejection.
  • agent-compose diff <left> <right> reports semantic decision changes rather than a raw filesystem diff.
  • Interactive compose prints a one-screen summary by default and offers an explicit full explanation.
  • Launch wrappers emit at most a one-line no-change success, expand on changes or warnings, and show full context on failure.
  • Chatty human output is kept out of generated agent instructions and is not accidentally injected into model context.
  • TTY output uses hierarchy and restrained color, while redirected output is plain, width-safe, and deterministic.
  • Stable machine-readable output exposes the same decision tree rather than a second explanation model.
  • The trace records facts, candidates, rules, outcomes, reasons, and normalized provenance without private content, credentials, opaque host values, or volatile absolute paths.
  • Runtime telemetry such as duration and cache location is visible but excluded from deterministic bundle identity.
  • agent-compose doctor links health failures to the relevant decision-tree node and gives one concrete recovery action.
  • Role and personality selection explains the person source, compatibility rule, allowed alternatives, and why the selected pairing is valid.
  • Capability, harness-delivery, fallback, cache-hit, and profile-difference fixtures exercise the explanation surface.
  • Tests assert decision semantics, ordering, redaction, and bounded default output without committing generated reference renders.

Blocked by

  • Blocked by #4
  • Blocked by #5

Execution type

HITL for terminal information design and terminology review. The trace model, rendering, redaction, and tests are otherwise AFK.

## Parent #1 ## What to build Make explainability a first-class product surface for agent-compose. Every resolution should produce a structured decision tree while choices are made, retain the deterministic evidence with the bundle, and render it for humans with progressive disclosure. The result should feel chatty and legible during interactive use without flooding unattended launch logs or model context. ## Human experience An interactive compose shows a bounded summary: resolved profile, target repos, cache result, changed outputs, selected source and skill counts, delivery mode, and warnings. A human can then expand the exact branch she cares about instead of paging through undifferentiated debug logs. The detailed tree should answer: * Which person source, agent, role, personality, model class, privacy scope, and repo facts were used? * Which policy rules were evaluated, in what order, and from which source? * Which source, capability, and skill candidates were considered? * Why was each candidate included, excluded, shadowed, or replaced? * Which fallback fired and what preferred input was unavailable? * Why was content mounted natively, compiled into instructions, or omitted? * Which decision changed between two bundles? ## Acceptance criteria - [ ] `agent-compose describe` renders the stored decision tree in stable, human-scannable sections. - [ ] `agent-compose describe --why <source-or-skill>` follows one item from repo facts through policy to its final delivery or rejection. - [ ] `agent-compose diff <left> <right>` reports semantic decision changes rather than a raw filesystem diff. - [ ] Interactive compose prints a one-screen summary by default and offers an explicit full explanation. - [ ] Launch wrappers emit at most a one-line no-change success, expand on changes or warnings, and show full context on failure. - [ ] Chatty human output is kept out of generated agent instructions and is not accidentally injected into model context. - [ ] TTY output uses hierarchy and restrained color, while redirected output is plain, width-safe, and deterministic. - [ ] Stable machine-readable output exposes the same decision tree rather than a second explanation model. - [ ] The trace records facts, candidates, rules, outcomes, reasons, and normalized provenance without private content, credentials, opaque host values, or volatile absolute paths. - [ ] Runtime telemetry such as duration and cache location is visible but excluded from deterministic bundle identity. - [ ] `agent-compose doctor` links health failures to the relevant decision-tree node and gives one concrete recovery action. - [ ] Role and personality selection explains the person source, compatibility rule, allowed alternatives, and why the selected pairing is valid. - [ ] Capability, harness-delivery, fallback, cache-hit, and profile-difference fixtures exercise the explanation surface. - [ ] Tests assert decision semantics, ordering, redaction, and bounded default output without committing generated reference renders. ## Blocked by - Blocked by #4 - Blocked by #5 ## Execution type HITL for terminal information design and terminology review. The trace model, rendering, redaction, and tests are otherwise AFK.
coilyco-ops changed title from Explain composition decisions and diagnose bundle health to Make composition decisions chatty and inspectable 2026-07-16 08:07:03 +00:00
coilyco-ops commented 2026-07-22 23:00:07 +00:00
Author
Member
Copy link

Shipped in fca3be3, built to the calibrated spec agreed in session (tens of lines, not hundreds).

Acceptance criteria:

  • describe renders the stored tree in stable sections (profile / sources / selection / delivery) with one line per decision; exclusion groups of 4+ collapse to a count line, --all expands. A full fixture bundle describes in ~20 lines.
  • describe --why follows one item: considered-by, outcome, reason, plus the counterfactual for excluded personality skills ('would select under: personality X (compatible with role Y)'). Accepts bare names or subject form.
  • diff is semantic, keyed by decision subject: outcome flips render as 'selected -> excluded', same-outcome reason changes as 'still excluded, reason changed', added/removed subjects as +/-, with an unchanged count and 'no semantic differences' for identical bundles.
  • compose keeps the 6-line summary; --explain appends the full tree.
  • Launch already emits one line on success, a warning block on fallback, full diagnostics on failure (shipped with #6).
  • Chatty output is stdout/stderr only and never enters bundle content; trace.json IS the machine-readable surface - no second explanation model.
  • Color (restrained: symbols only) renders only on a TTY with NO_COLOR unset; redirected output is plain and deterministic (asserted in tests).
  • Role and personality pairing now leaves trace evidence: two profile decisions record the person source, the role purpose, and the compatible personality set.

Trimmed against the original text, as discussed: doctor deferred (every failure path already names cause and fix; revisit at v0.2), and the repo-facts/privacy-scope/model-class explanation bullets fell with the #13 review.

Closed by Claude Code working the issue queue.

Shipped in fca3be3, built to the calibrated spec agreed in session (tens of lines, not hundreds). Acceptance criteria: - describe renders the stored tree in stable sections (profile / sources / selection / delivery) with one line per decision; exclusion groups of 4+ collapse to a count line, --all expands. A full fixture bundle describes in ~20 lines. - describe --why <item> follows one item: considered-by, outcome, reason, plus the counterfactual for excluded personality skills ('would select under: personality X (compatible with role Y)'). Accepts bare names or subject form. - diff <left> <right> is semantic, keyed by decision subject: outcome flips render as 'selected -> excluded', same-outcome reason changes as 'still excluded, reason changed', added/removed subjects as +/-, with an unchanged count and 'no semantic differences' for identical bundles. - compose keeps the 6-line summary; --explain appends the full tree. - Launch already emits one line on success, a warning block on fallback, full diagnostics on failure (shipped with #6). - Chatty output is stdout/stderr only and never enters bundle content; trace.json IS the machine-readable surface - no second explanation model. - Color (restrained: symbols only) renders only on a TTY with NO_COLOR unset; redirected output is plain and deterministic (asserted in tests). - Role and personality pairing now leaves trace evidence: two profile decisions record the person source, the role purpose, and the compatible personality set. Trimmed against the original text, as discussed: doctor deferred (every failure path already names cause and fix; revisit at v0.2), and the repo-facts/privacy-scope/model-class explanation bullets fell with the #13 review. Closed by Claude Code working the issue queue.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
feat/gamedev-role
feat/boundary-omit
aos/claude/eval-context
aos/claude/eval-current-state
aos/claude/eval-refs-restore
aos/claude/eval-docs-one-page
aos/claude/personality-anchors
aos/claude/favorite-color-spread
aos/claude/personality-catalog-gaps
aos/claude/gh297-binding-skill-selector
aos/claude/personality-prose-distinctiveness
aos/claude/gh56-adopt-aos-eval
aos/claude/aw85-bands
aos/claude/xu59-just
aos/claude/xu59-eval-export
aos/claude/xu59-comment-caps
aos/claude/xu59-pr-ci
aos/claude/xu59-role-attribution
aos/claude/ck78-banner
aos/claude/mg96-fm
rescue/precommit-v0.18-uncommitted
evals/262-board-run
claude/agents-temp-clone-note
assets/compose-mark
adjacency/262-role-adjacents
chore/precommit-bump
prose/257-258-233-second-person
boundary/254-primitive-rename
fix/acompose-undefined-role-bindings
comms/rescope-counterpart
native-home/968-user-scope-projection
meld/248-evidence-acquisition
meld/231-shared-role-melds
evals/240-baseline
evals/240-model-arms
spike/claude-native-ui-surfaces
issue-225-compact-director-eval
issue-210
issue-218
issue-215-remove-context-levels
codex/luna-medium-ai-evaluation
issue-206-align-role-tiers
codex/luna-medium-roster-prose
issue-196-remove-legacy-role-providers
issue-197-role-provider-graph
feat/model-tiers
feat/remove-context-levels
issue-187
issue-178-content-communication-ownership
issue-163-designer-evals
codex/issue-794-preserve-canonical-projections
issue-163-ornith-direct-eval
issue-163-design-current-eval
issue-150-v2-baseline
issue-156-content-code
issue-150-v2-evidence
issue-150-v2-evaluation
issue-154-v2-main-integration
issue-148-remove-v1-integrations
issue-146-core-roster
issue-144-aos-catalogue-manifest
issue-152-release-classifier
issue-139-role-metadata-last
issue-138-designer-visual-web
issue-785-first-person
ward-salvage/agent-compose-078db83c
ward-salvage/agent-compose-74e37165
issue-130
recovery/2026-07-27-local-work
v2.39.0
v2.38.0
v2.37.0
v2.36.0
v2.35.0
v2.34.0
v2.33.0
v2.32.0
v2.31.0
v2.30.0
v2.29.0
v2.28.0
v2.27.0
v2.26.0
v2.25.0
v2.24.0
v2.23.0
v2.22.0
v2.21.0
v2.20.0
v2.19.0
v2.18.0
v2.17.0
v2.16.0
v2.15.0
v2.14.0
v2.13.0
v2.12.0
v2.11.0
v2.10.0
v2.9.0
v2.8.0
v2.7.0
v2.6.0
v2.5.0
v2.4.0
v2.3.0
v2.2.0
v2.1.0
v2.0.0
v1.49.0
v1.48.0
v1.47.0
v1.46.0
v1.45.0
v1.44.0
v1.43.0
v1.42.0
v1.41.0
v1.40.0
v1.39.0
v1.38.0
v1.37.0
v1.36.0
v1.35.0
v1.34.0
v1.33.0
v1.32.0
v1.31.0
v1.30.0
v1.29.0
v1.28.0
v1.27.0
v1.26.0
v1.25.0
v1.24.0
v1.23.0
v1.22.0
v1.21.0
v1.20.0
v1.19.0
v1.18.0
v1.17.0
v1.16.0
v1.15.0
v1.14.0
v1.13.0
v1.12.0
v1.11.0
v1.10.0
v1.9.0
v1.8.0
v1.7.0
v1.6.0
v1.5.0
v1.4.0
v1.3.0
v1.2.0
v1.1.0
v1.0.0
v0.70.0
v0.69.0
v0.68.0
v0.67.0
v0.66.0
v0.65.0
v0.64.0
v0.63.0
v0.62.0
v0.61.0
v0.60.0
v0.59.0
v0.58.0
v0.57.0
v0.56.0
v0.55.0
v0.54.0
v0.53.0
v0.52.0
v0.51.0
v0.50.0
v0.49.0
v0.48.0
v0.47.0
v0.46.0
v0.45.0
v0.44.0
v0.43.0
v0.42.0
v0.41.0
v0.40.0
v0.39.0
v0.38.0
v0.37.0
v0.36.0
v0.35.0
v0.34.0
v0.33.0
v0.32.0
v0.31.0
v0.30.0
v0.29.0
v0.28.0
v0.27.0
v0.26.0
v0.25.0
v0.24.0
v0.23.0
v0.22.0
v0.21.0
v0.20.0
v0.19.0
v0.18.0
v0.17.0
v0.16.0
v0.15.0
v0.14.0
v0.13.0
v0.12.0
v0.11.0
v0.10.0
v0.9.0
v0.8.0
v0.7.0
v0.6.0
v0.5.0
v0.4.1
v0.4.0
v0.3.0
v0.2.1
v0.2.0
v0.1.2
v0.1.1
v0.1.0
Labels
Clear labels
  
autonomy
async-consult
 A human needs to consult on the issue to upgrade it to headless

  
autonomy
epic
 This issue has many units of sub work - its size makes it meaningfully exclusive with other autonomy types

  
autonomy
headless
 The agent can perform the work on its own

  
autonomy
live-collab
 The agent and the human need to work together in realtime

  
coherence-core
Core review set for the warded control plane coherence milestone. These issues form the release spine; adjacent milestone issues are stretch or supporting work.

  
priority
P0
 priority tier

  
priority
P1
 priority tier

  
priority
P2
 priority tier

  
priority
P3
 priority tier

  
priority
P4
 priority tier

  
qa-fixture
Disposable issue admitted to the bounded Ward QA verification lane.

  
role/ai
requires work from the AI Engineer role

  
role/creator
requires work from Content Creator role

  
role/design
requires work from the design role

  
role/director
requires work from the director role

  
role/engineer
requires work from the engineer role

  
role/exec
requires work from the exec role

  
role/human
requires a person, and specifically not an agent seat

  
role/ops
requires work from the ops role

  
role/qa
requires work from the QA role

No labels
autonomy
async-consult
autonomy
epic
autonomy
headless
autonomy
live-collab
coherence-core
priority
P0
priority
P1
priority
P2
priority
P3
priority
P4
qa-fixture
role/ai
role/creator
role/design
role/director
role/engineer
role/exec
role/human
role/ops
role/qa
Milestone
Clear milestone
No items
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
coilyco-flight-deck/agent-compose#8
No description provided.