Skip to content

Spike: instruction resolver with most-specificity rule#1867

Draft
SamMorrowDrums wants to merge 1 commit into
mainfrom
sammorrowdrums/spike-instruction-specificity
Draft

Spike: instruction resolver with most-specificity rule#1867
SamMorrowDrums wants to merge 1 commit into
mainfrom
sammorrowdrums/spike-instruction-specificity

Conversation

@SamMorrowDrums

Copy link
Copy Markdown
Collaborator

Summary

Exploratory spike to evaluate finer-grained instruction control by associating instructions with specific sets of tools rather than whole toolsets.

Design

Core Types

  • InstructionRule - Associates an instruction with a set of tool names
  • InstructionResolver - Resolves which instructions apply using the most-specificity rule

Specificity Algorithm

  1. A rule matches when ALL its tools are present in active tools
  2. When Rule A's tools are a proper superset of Rule B's tools (A ⊃ B), A shadows B
  3. Partial overlaps (neither is superset) → both rules apply
  4. Output is sorted for determinism

Example

rules := []InstructionRule{
    NewInstructionRule("issues-all", "All issues instruction", "get_issue", "list_issues", "create_issue"),
    NewInstructionRule("issues-read", "Read issues instruction", "get_issue", "list_issues"),
    NewInstructionRule("create-only", "Create only instruction", "create_issue"),
}

// Active tools: {get_issue, list_issues, create_issue}
// Result: Only "issues-all" applies (shadows the other two)

// Active tools: {get_issue, list_issues}  
// Result: Only "issues-read" applies

Testing

All tests pass:

  • TestInstructionResolver_BasicMatching
  • TestInstructionResolver_SupersetShadowing
  • TestInstructionResolver_PartialOverlapNoShadowing
  • TestInstructionResolver_ComplexHierarchy
  • TestInstructionResolver_EmptyRules
  • TestInstructionResolver_EmptyActiveTools

Next Steps (if we proceed)

  • Integrate with existing GenerateInstructions() or create a V2
  • Define actual instruction rules based on tool combinations
  • Consider adding priority/weight for equal-size rule conflicts
Add InstructionRule and InstructionResolver types that enable associating
instructions with specific sets of tools rather than whole toolsets.

Key features:
- Rules match when ALL specified tools are present in active tools
- Most-specific rule wins: superset rules shadow subset rules
- Partial overlaps (neither superset) result in both rules applying
- Deterministic output via sorted results

This is exploratory work to evaluate finer-grained instruction control.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant