firefrost-gaming/skill-seekers-reference
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
34 Commits 1 Branch 0 Tags
e802dfee6d21e87aa8f6538290ec1d01995d7ebb
Commit Graph

6 Commits

Author SHA1 Message Date
yusyus
bddb57f5ef Add large documentation handling (40K+ pages support) 
Implement comprehensive system for handling very large documentation sites
with intelligent splitting strategies and router/hub architecture.

**New CLI Tools:**
- cli/split_config.py: Split large configs into focused sub-skills
  * Strategies: auto, category, router, size
  * Configurable target pages per skill (default: 5000)
  * Dry-run mode for preview

- cli/generate_router.py: Create intelligent router/hub skills
  * Auto-generates routing logic based on keywords
  * Creates SKILL.md with topic-to-skill mapping
  * Infers router name from sub-skills

- cli/package_multi.py: Batch package multiple skills
  * Package router + all sub-skills in one command
  * Progress tracking for each skill

**MCP Integration:**
- Added split_config tool (8 total MCP tools now)
- Added generate_router tool
- Supports 40K+ page documentation via MCP

**Configuration:**
- New split_strategy parameter in configs
- split_config section for fine-tuned control
- checkpoint section for resume capability (ready for Phase 4)
- Example: configs/godot-large-example.json

**Documentation:**
- docs/LARGE_DOCUMENTATION.md (500+ lines)
  * Complete guide for 10K+ page documentation
  * All splitting strategies explained
  * Detailed workflows with examples
  * Best practices and troubleshooting
  * Real-world examples (AWS, Microsoft, Godot)

**Features:**
✅ Handle 40K+ page documentation efficiently
✅ Parallel scraping support (5x-10x faster)
✅ Router + sub-skills architecture
✅ Intelligent keyword-based routing
✅ Multiple splitting strategies
✅ Full MCP integration

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-10-19 20:48:03 +03:00
yusyus
f103aa62cb Clean up tracked files and repository structure 
Remove unnecessary files:
- configs/.DS_Store (macOS system file, should not be tracked)

This ensures only relevant project files are version controlled
and improves repository hygiene.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-10-19 19:45:13 +03:00
yusyus
d7e6142ab0 Add test configurations for MCP validation 
Add 4 test configuration files used for validating MCP functionality:
- astro.json: Astro framework documentation (15 pages, production test)
- python-tutorial-test.json: Python tutorial (minimal test case)
- tailwind.json: Tailwind CSS documentation (test case)
- test-manual.json: Manual testing configuration

These configs were used to verify:
- Config generation via generate_config tool
- Config validation via validate_config tool
- Page estimation via estimate_pages tool
- Full scraping workflow via scrape_docs tool
- Skill packaging via package_skill tool

All tests passed successfully in production Claude Code environment.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-10-19 19:44:27 +03:00
jarek
7a4c1d7083 kubernetes config for official docs 2025-10-19 09:28:44 +02:00
yusyus
59c2f9126d Optimize all framework configs with start_urls for better coverage 
All configs now follow the steam-economy-complete.json pattern with:
- Multiple start_urls for comprehensive entry points
- Improved include patterns for better targeting
- Enhanced exclude patterns to skip irrelevant pages

Godot Config:
- Added 7 start_urls covering getting started, scripting, 2D, 3D, physics, animation, and classes
- Added include patterns: /getting_started/, /tutorials/, /classes/
- More focused scraping of core documentation

React Config:
- Added 6 start_urls covering learn, quick-start, reference, and hooks
- Existing patterns maintained (already well-optimized)

Vue Config:
- Added 6 start_urls covering introduction, essentials, components, composables, and API
- Fixed base_url from https://vuejs.org/guide/ to https://vuejs.org/
- Added /partners/ to exclude list

Django Config:
- Added 7 start_urls covering intro, models, views, templates, forms, auth, and reference
- Added /intro/ to include patterns
- Added /releases/ to exclude list (changelog not needed)

FastAPI Config:
- Added 7 start_urls covering tutorial, first-steps, path-params, body, dependencies, advanced, and reference
- Added /deployment/ to exclude list

Benefits:
- Better initial page discovery
- More comprehensive documentation coverage
- Faster scraping (direct entry to important sections)
- Reduced unnecessary page crawling
- Consistent pattern across all configs

All configs tested and validated:
✅ 71/71 tests passing
✅ All 6 configs validated successfully

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-10-19 02:24:56 +03:00
yusyus
78b9cae398 Init 2025-10-17 15:14:44 +00:00