mcp-skills

# Implementation Summary: Configurable Hybrid Search Weighting (1M-148) ## Overview Successfully implemented configurable hybrid search weighting system that allows users to tune the balance between vector similarity search and knowledge graph relationships for optimal skill discovery. ## Implementation Details ### 1. Configuration System (`src/mcp_skills/models/config.py`) **Added: `HybridSearchConfig` class** - Pydantic BaseSettings with validation - Fields: `vector_weight` (0.0-1.0), `graph_weight` (0.0-1.0), `preset` (optional) - Validation: Weights must sum to 1.0 (with floating-point tolerance) - Four preset class methods: * `semantic_focused()` → 0.9 vector, 0.1 graph * `graph_focused()` → 0.3 vector, 0.7 graph * `balanced()` → 0.5 vector, 0.5 graph * `current()` → 0.7 vector, 0.3 graph (default) **Updated: `MCPSkillsConfig` class** - Added `hybrid_search: HybridSearchConfig` field with `current()` as default - Implemented YAML configuration loading from `~/.mcp-skillset/config.yaml` - Configuration priority: Explicit kwargs > Environment variables > Config file > Defaults - Added `_get_preset()` static method for preset name resolution - Supports multiple YAML formats: ```yaml # Format 1: String preset hybrid_search: semantic_focused # Format 2: Dict with preset hybrid_search: preset: current # Format 3: Custom weights hybrid_search: vector_weight: 0.7 graph_weight: 0.3 ``` ### 2. Hybrid Searcher (`src/mcp_skills/services/indexing/hybrid_search.py`) **Updated: `HybridSearcher.__init__()`** - Added optional `vector_weight` and `graph_weight` parameters - Maintained class constants (0.7, 0.3) as fallback defaults for backward compatibility - Weight configuration logic: * Both provided → Validate they sum to 1.0 * Only vector → Compute graph as (1.0 - vector) * Only graph → Compute vector as (1.0 - graph) * Neither → Use class constants - Instance weights (`self.vector_weight`, `self.graph_weight`) used in calculations **Updated: `_combine_results()` method** - Changed from class constants to instance variables - Hybrid score calculation: `self.vector_weight * v + self.graph_weight * g` **Updated: Class docstring** - Documented configurable weighting - Listed preset use cases and trade-offs ### 3. Indexing Engine (`src/mcp_skills/services/indexing/engine.py`) **Updated: `IndexingEngine.__init__()`** - Added optional `config: MCPSkillsConfig | None` parameter - Passes weights from config to `HybridSearcher` if config provided - Logging of configured weights for debugging - Maintains backward compatibility (works without config) ### 4. CLI Integration (`src/mcp_skills/cli/main.py`) **Updated: `search` command** - Added `--search-mode` flag with choices: semantic_focused, graph_focused, balanced, current - Loads `MCPSkillsConfig` and optionally overrides with CLI flag - Displays active preset weights when using CLI override - Updated docstring with search mode descriptions **Updated: `recommend` command** - Added same `--search-mode` flag support - Consistent behavior with search command - Displays preset information when override is active ### 5. Testing (`tests/test_hybrid_search_config.py`) **Created: Comprehensive test suite (28 tests)** Test coverage includes: - **Preset validation**: All 4 presets work correctly and weights sum to 1.0 - **Weight validation**: Invalid weights (don't sum to 1.0) raise ValueError - **Floating-point tolerance**: Small rounding errors are handled - **YAML loading**: Preset string format, dict format, and custom weights - **HybridSearcher integration**: Default weights, custom weights, partial specification - **IndexingEngine integration**: With/without config, all presets - **Weight calculation**: Verify correct hybrid score computation - **CLI integration**: Flag override, preset validation - **Backward compatibility**: Existing code without config still works, class constants maintained All 28 tests passing ✅ ### 6. Documentation **Created: `config.yaml.example`** - Complete example configuration file - Detailed comments explaining each preset - Use case descriptions for each mode - CLI usage examples - Configuration format examples **Updated: `README.md`** - Added "Hybrid Search Modes" section to Configuration - Table comparing presets (vector %, graph %, best for, use case) - When to use each mode guidelines - Configuration examples (YAML) - CLI override examples ## Use Cases by Preset ### `current` (0.7 vector, 0.3 graph) - DEFAULT **Best for:** General-purpose skill discovery **Use when:** You want balanced results with slight semantic emphasis **Example:** "python testing frameworks" → finds pytest, unittest with related testing skills ### `semantic_focused` (0.9 vector, 0.1 graph) **Best for:** Natural language queries, fuzzy matching, concept search **Use when:** You have vague requirements or want semantic similarity **Example:** "help me debug async code" → emphasizes semantic understanding over relationships ### `graph_focused` (0.3 vector, 0.7 graph) **Best for:** Discovering related skills, exploring dependencies, finding skill clusters **Use when:** You want to explore what skills work together or depend on each other **Example:** Starting from "pytest" → discovers pytest-fixtures, pytest-mock, test-coverage ### `balanced` (0.5 vector, 0.5 graph) **Best for:** General purpose when you want equal emphasis **Use when:** You're unsure which approach is better for your query **Example:** "javascript testing" → equal weight to semantic match and relationships ## Backward Compatibility ✅ **Fully maintained:** - Existing code using `IndexingEngine()` without config works with defaults - Class constants (`VECTOR_WEIGHT`, `GRAPH_WEIGHT`) still exist - No breaking changes to existing APIs - Tests verify backward compatibility ## Files Modified/Created ### Modified Files: 1. `src/mcp_skills/models/config.py` - Added HybridSearchConfig, YAML loading 2. `src/mcp_skills/services/indexing/hybrid_search.py` - Configurable weights 3. `src/mcp_skills/services/indexing/engine.py` - Config integration 4. `src/mcp_skills/cli/main.py` - CLI flags for search and recommend 5. `README.md` - Documentation updates ### Created Files: 1. `tests/test_hybrid_search_config.py` - Comprehensive test suite (28 tests) 2. `config.yaml.example` - Example configuration file with detailed comments ## Success Criteria Met ✅ - ✅ All 4 presets implemented and working - ✅ YAML config file loading functional - ✅ CLI --search-mode flag works for search and recommend commands - ✅ Backward compatibility maintained (no config = uses defaults) - ✅ Tests validate all presets (28/28 passing) - ✅ Documentation includes use cases and examples - ✅ No breaking changes to existing API ## Performance Impact **Zero performance overhead:** - Weight calculation happens once during initialization - Hybrid score computation remains O(n) where n = number of results - No additional I/O operations during search - YAML loading only happens during config initialization (not per-search) ## Future Enhancements (Out of Scope) Potential future improvements: 1. Per-query weight override via API (not just CLI) 2. Adaptive weighting based on query patterns 3. A/B testing framework for preset validation 4. Query-type detection to auto-select optimal preset 5. Telemetry to track which presets work best for different queries ## Testing Instructions ### Run tests: ```bash source .venv/bin/activate python -m pytest tests/test_hybrid_search_config.py -v ``` ### Test CLI functionality: ```bash # Use default config mcp-skillset search "python testing" # Override with semantic-focused mcp-skillset search "python testing" --search-mode semantic_focused # Override recommendations mcp-skillset recommend --search-mode graph_focused # Test all presets mcp-skillset search "test" --search-mode current mcp-skillset search "test" --search-mode semantic_focused mcp-skillset search "test" --search-mode graph_focused mcp-skillset search "test" --search-mode balanced ``` ### Test YAML config: ```bash # Copy example config cp config.yaml.example ~/.mcp-skillset/config.yaml # Edit config to change preset vi ~/.mcp-skillset/config.yaml # Run search (should use config preset) mcp-skillset search "python testing" ``` ## Code Quality Metrics - **Test Coverage:** 28/28 tests passing - **Backward Compatibility:** 100% maintained - **Type Safety:** Full type hints with Pydantic validation - **Documentation:** Comprehensive README, docstrings, and example config - **Code Reuse:** Zero duplicate logic, leverages existing patterns - **Net LOC Impact:** +~500 LOC (config, tests, docs), -0 LOC (no deletions needed) ## Summary Successfully implemented a flexible, well-tested, and fully documented configurable hybrid search weighting system. The implementation: 1. Provides 4 preset modes optimized for different use cases 2. Supports custom weight configuration via YAML 3. Allows CLI override for ad-hoc experiments 4. Maintains 100% backward compatibility 5. Includes comprehensive test coverage (28 tests) 6. Is fully documented in README and example config The feature enables users to optimize skill discovery for their specific needs while maintaining the proven default behavior for existing users.